Apr 21 13:29:23.167718 np0000168688 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service.
Apr 21 13:29:23.487612 np0000168688 neutron-ovn-maintenance-worker[105483]: /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:41: EventletDeprecationWarning:
Apr 21 13:29:23.487612 np0000168688 neutron-ovn-maintenance-worker[105483]: Eventlet is deprecated. It is currently being maintained in bugfix mode, and
Apr 21 13:29:23.487612 np0000168688 neutron-ovn-maintenance-worker[105483]: we strongly recommend against using it for new projects.
Apr 21 13:29:23.487612 np0000168688 neutron-ovn-maintenance-worker[105483]: If you are already using Eventlet, we recommend migrating to a different
Apr 21 13:29:23.487612 np0000168688 neutron-ovn-maintenance-worker[105483]: framework.  For more detail see
Apr 21 13:29:23.487612 np0000168688 neutron-ovn-maintenance-worker[105483]: https://eventlet.readthedocs.io/en/latest/asyncio/migration.html
Apr 21 13:29:23.487612 np0000168688 neutron-ovn-maintenance-worker[105483]:   import eventlet
Apr 21 13:29:24.547277 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.common.config [-] Logging enabled!
Apr 21 13:29:24.547277 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev136
Apr 21 13:29:24.548093 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=105483) setup_logging /opt/stack/neutron/neutron/common/config.py:124}}
Apr 21 13:29:24.548093 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting...
Apr 21 13:29:24.548093 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=105483) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Apr 21 13:29:24.548093 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=105483) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Apr 21 13:29:24.548093 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=105483) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Apr 21 13:29:24.548093 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=105483) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Apr 21 13:29:24.548093 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=105483) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Apr 21 13:29:24.548093 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.manager [-] Loading core plugin: ml2
Apr 21 13:29:24.814255 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.814412 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.814412 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.814974 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.815048 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.815127 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.852204 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.852204 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.852204 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.858482 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.858557 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.858661 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.996764 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.996912 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:24.996912 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.008361 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.008538 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.008538 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.055883 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7d4d543634c0> agent after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.056010 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7d4d543634c0> agent after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.056010 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _add_segment_host_mapping_for_segment at 0x7d4d54363560> segment precommit_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.056010 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _delete_segments_for_network at 0x7d4d54363600> network precommit_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.056309 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _prevent_segment_delete_with_port_bound at 0x7d4d5438c860> segment before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.107754 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._before_network_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> network before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.107897 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function DVRDbMixin._delete_mac_associated_with_agent at 0x7d4d5454c680> agent before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.107897 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> network before_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108040 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> port before_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108040 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> port before_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108175 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> segment precommit_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108175 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> segment precommit_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108175 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> segment after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108422 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> segment after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108422 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> network after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108422 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> network precommit_delete 0, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108620 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> port provisioning_complete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108620 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_create of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> rbac-policy before_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108620 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_delete of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> rbac-policy after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108804 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._retry_binding_revived_agents of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> agent after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108804 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> subnet after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108804 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> subnet precommit_delete 0, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108989 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> rbac-policy before_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108989 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> rbac-policy before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.108989 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> segment_host_mapping after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.109172 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin.delete_agent_notified of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> agent after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.109172 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> rbac-policy before_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.109172 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> rbac-policy before_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.109418 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7d4d559bc6b0>> rbac-policy before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.109637 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=105483) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 21 13:29:25.109637 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=105483) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 21 13:29:25.109772 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=105483) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 21 13:29:25.109772 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=105483) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 21 13:29:25.109772 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=105483) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 21 13:29:25.109933 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=105483) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 21 13:29:25.110028 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve']
Apr 21 13:29:25.110787 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public']
Apr 21 13:29:25.111396 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
Apr 21 13:29:25.112133 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]})
Apr 21 13:29:25.112234 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan']
Apr 21 13:29:25.112342 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan'])
Apr 21 13:29:25.112445 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve']
Apr 21 13:29:25.112564 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos']
Apr 21 13:29:25.113073 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos']
Apr 21 13:29:25.113146 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos']
Apr 21 13:29:25.113245 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn']
Apr 21 13:29:25.152449 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn']
Apr 21 13:29:25.152588 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn']
Apr 21 13:29:25.152588 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling.
Apr 21 13:29:25.153122 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7d4d540aaae0>> floatingip before_response 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.153267 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7d4d540aaae0>> port before_response 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.174785 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat'
Apr 21 13:29:25.174785 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete
Apr 21 13:29:25.174945 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve'
Apr 21 13:29:25.174982 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)]
Apr 21 13:29:25.221105 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO dbcounter [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Registered counter for database neutron
Apr 21 13:29:25.234561 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG oslo_db.sqlalchemy.engines [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105483) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}}
Apr 21 13:29:25.642688 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG dbcounter [-] [105483] Writer thread running {{(pid=105483) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
Apr 21 13:29:25.832369 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Initializing driver for type 'local'
Apr 21 13:29:25.832523 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Initializing driver for type 'vlan'
Apr 21 13:29:25.952435 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] VlanTypeDriver initialization complete
Apr 21 13:29:25.952435 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Initializing extension driver 'port_security'
Apr 21 13:29:25.952721 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.extensions.port_security [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] PortSecurityExtensionDriver initialization complete
Apr 21 13:29:25.952721 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Initializing extension driver 'qos'
Apr 21 13:29:25.952721 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.plugins.ml2.extensions.qos [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QosExtensionDriver initialization complete {{(pid=105483) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}}
Apr 21 13:29:25.952721 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Initializing mechanism driver 'ovn'
Apr 21 13:29:25.952949 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Starting OVNMechanismDriver
Apr 21 13:29:25.954246 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver.pre_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> process before_spawn 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.954371 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver.post_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> process after_init 55550000, True {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.954371 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._add_segment_host_mapping_for_segment of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> segment after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.954489 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver.create_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> segment after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.954546 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver.delete_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> segment after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.954623 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._validate_allowed_address_pairs of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> allowed_address_pair before_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.954864 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> security_group precommit_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.954864 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._update_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> security_group after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.954864 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> security_group after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.955024 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> security_group precommit_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.955024 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> security_group after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.955140 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._create_sg_rule_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> security_group_rule precommit_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.955260 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> security_group_rule after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.955260 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> security_group_rule before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.955373 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> address_group after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.955435 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> address_group after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.955533 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7d4d540bcce0>> address_group after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.955656 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method DriverBase._register of <neutron.services.qos.drivers.ovn.driver.OVNQosDriver object at 0x7d4d53d62a80>> qos_plugin after_init 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.955827 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNTrunkDriver.register of <neutron.services.trunk.drivers.ovn.trunk_driver.OVNTrunkDriver object at 0x7d4d54215970>> trunk_plugin after_init 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.955990 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method DriverBase._register of <neutron.services.logapi.drivers.ovn.driver.OVNDriver object at 0x7d4d53348b00>> logging-plugin after_init 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.956378 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.services.logapi.drivers.ovn.driver [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] OVN logging driver registered
Apr 21 13:29:25.957556 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.plugins.ml2.plugin [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=105483) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
Apr 21 13:29:25.957764 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348d70>> port after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.957839 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348d70>> port after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.957919 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348d70>> port after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.957997 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348ce0>> subnet after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958088 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348ce0>> subnet after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958151 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348ce0>> subnet after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958233 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348c80>> network after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958329 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348c80>> network after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958393 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348c80>> network after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958469 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348c50>> security_group after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958541 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348c50>> security_group after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958643 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348c50>> security_group after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958710 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348cb0>> security_group_rule after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958770 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348cb0>> security_group_rule after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958852 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348cb0>> security_group_rule after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958927 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348bc0>> address_group after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.958998 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348bc0>> address_group after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.959059 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7d4d53348bc0>> address_group after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.959126 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ML2 OVO RPC backend initialized. {{(pid=105483) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}}
Apr 21 13:29:25.959819 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.quota [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x7d4d53348b90>.
Apr 21 13:29:25.959909 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.plugin [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Modular L2 Plugin initialization complete
Apr 21 13:29:25.959968 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading services supported by the core plugin {{(pid=105483) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}}
Apr 21 13:29:25.960081 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.plugins.ml2.managers [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Got port-security extension from driver 'port_security'
Apr 21 13:29:25.960314 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=105483) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}}
Apr 21 13:29:25.960405 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading Plugin: ovn-router
Apr 21 13:29:25.967451 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._delete_router_remove_external_gateways of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7d4d5330e0c0>> router before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.967561 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <function L3RpcNotifierMixin._notify_gateway_port_ip_changed at 0x7d4d541cc5e0> port after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.967625 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <function L3RpcNotifierMixin._notify_routers_callback at 0x7d4d541cc4a0> port after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.967709 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <function L3RpcNotifierMixin._notify_subnet_gateway_ip_update at 0x7d4d541cc540> subnet after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.967794 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <function L3RpcNotifierMixin._notify_subnetpool_address_scope_update at 0x7d4d541cc680> subnetpool_address_scope after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.967856 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7d4d53323e20> port before_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.967934 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7d4d53323e20> port after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.968040 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method L3_NAT_dbonly_mixin._precommit_delete_port_callback of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7d4d5330e0c0>> port precommit_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.968105 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_internal_ip_change_for_fip at 0x7d4d541b4540> port before_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.968162 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback at 0x7d4d541b44a0> port before_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.968293 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method RouterAvailabilityZoneMixin._process_az_request of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7d4d5330e0c0>> router precommit_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.968364 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_create of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7d4d5330e0c0>> router precommit_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.968445 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_update of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7d4d5330e0c0>> router precommit_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.968508 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <function OVNL3RouterPlugin._subnet_update at 0x7d4d53323d80> subnet after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.969158 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.quota.resource_registry [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Tracking information for resource: router configured {{(pid=105483) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 21 13:29:25.969285 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.quota.resource_registry [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Tracking information for resource: floatingip configured {{(pid=105483) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 21 13:29:25.969285 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.services.ovn_l3.plugin [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Starting OVNL3RouterPlugin
Apr 21 13:29:25.969489 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNPortForwarding.register of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7d4d5330d550>> port_forwarding_plugin after_init 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.969648 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method DriverController._check_router_request of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7d4d5330da30>> router before_create 55550100, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.969738 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method DriverController._clear_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7d4d5330da30>> router precommit_delete 55550100, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.969803 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method DriverController._set_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7d4d5330da30>> router precommit_create 55550100, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.969884 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method DriverController._update_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7d4d5330da30>> router precommit_update 55550100, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.972324 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.services.provider_configuration [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Service providers = [] {{(pid=105483) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:170}}
Apr 21 13:29:25.972431 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.db.servicetype_db [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=105483) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:42}}
Apr 21 13:29:25.973464 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVN_L3_HA_db_mixin._precommit_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> router precommit_create 55549900, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.973551 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_add_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> router_interface after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.973630 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_floatingip_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> floatingip after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.973717 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> floatingip precommit_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.973792 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> floatingip precommit_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.973878 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_floatingip_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> floatingip after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.973954 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_floatingip_status_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> floatingip after_status_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.974033 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_floatingip_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> floatingip after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.974111 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_portforwarding_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> port_forwarding after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.974191 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_remove_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> router_interface after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.974289 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> router after_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.974370 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_router_create_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> router precommit_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.974448 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_router_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> router after_delete 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.974528 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OvnDriver._process_router_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7d4d5334ad80>> router after_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.974618 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.services.service_base [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=105483) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}}
Apr 21 13:29:25.974713 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <bound method OVNL3RouterPlugin._post_fork_initialize of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7d4d5330e0c0>> process after_init 55550001, True {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.974817 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=105483) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 21 13:29:25.974874 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading Plugin: auto_allocate
Apr 21 13:29:25.975618 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7d4d533320c0> network precommit_update 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.975699 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron_lib.callbacks.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7d4d533320c0> network precommit_create 55550000, False {{(pid=105483) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:25.975872 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=105483) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 21 13:29:25.975927 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading Plugin: tag
Apr 21 13:29:25.978078 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=105483) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 21 13:29:25.978144 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading Plugin: timestamp
Apr 21 13:29:25.979146 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=105483) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 21 13:29:25.979232 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading Plugin: network_ip_availability
Apr 21 13:29:25.980400 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=105483) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 21 13:29:25.980467 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading Plugin: flavors
Apr 21 13:29:25.981227 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=105483) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 21 13:29:25.981309 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading Plugin: revisions
Apr 21 13:29:25.982092 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.manager [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=105483) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 21 13:29:25.982171 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG oslo_concurrency.lockutils [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.434s {{(pid=105483) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Apr 21 13:29:25.982374 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=105483) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}}
Apr 21 13:29:25.982446 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Initializing extension manager.
Apr 21 13:29:25.982664 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: __init__.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.982731 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: __pycache__ {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.982788 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: address_group.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.983075 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.983289 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: address-group
Apr 21 13:29:25.983358 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: address_scope.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.983598 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.983741 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: address-scope
Apr 21 13:29:25.983803 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: admin_state_down_before_update.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.983997 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.984138 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins
Apr 21 13:29:25.984241 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: agent.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.984456 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.984601 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: agent
Apr 21 13:29:25.984700 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: agent_resources_synced.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.984904 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.985027 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension agent-resources-synced not supported by any of loaded plugins
Apr 21 13:29:25.985115 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: allowedaddresspairs.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.985649 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.985790 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: allowed-address-pairs
Apr 21 13:29:25.985885 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: auto_allocated_topology.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.986058 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.986197 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: auto-allocated-topology
Apr 21 13:29:25.986285 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: availability_zone.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.986503 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.986637 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: availability_zone
Apr 21 13:29:25.986703 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: availability_zone_filter.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.986917 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.987035 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension availability_zone_filter not supported by any of loaded plugins
Apr 21 13:29:25.987136 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: data_plane_status.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.987288 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.987424 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension data-plane-status not supported by any of loaded plugins
Apr 21 13:29:25.987491 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: default_subnetpools.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.987670 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.987805 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: default-subnetpools
Apr 21 13:29:25.987866 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: dhcpagentscheduler.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.988170 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.988309 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: dhcp_agent_scheduler
Apr 21 13:29:25.988373 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: dns.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.988567 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.988772 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: dns-integration
Apr 21 13:29:25.988772 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: dns_domain_ports.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.988969 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.989096 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: dns-domain-ports
Apr 21 13:29:25.989158 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.989368 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like <project_id>, <project_name>, <user_id> and <user_name> as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.989487 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: dns-integration-domain-keywords
Apr 21 13:29:25.989560 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: dvr.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.989768 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.989896 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension dvr not supported by any of loaded plugins
Apr 21 13:29:25.989966 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: ecmp_routes.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.990142 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.990294 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension ecmp_routes not supported by any of loaded plugins
Apr 21 13:29:25.990363 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: empty_string_filtering.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.990536 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.990669 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: empty-string-filtering
Apr 21 13:29:25.990738 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.990917 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.991048 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins
Apr 21 13:29:25.991117 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.991310 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.991452 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins
Apr 21 13:29:25.991509 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: external_net.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.991692 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.991825 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: external-net
Apr 21 13:29:25.991893 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: extra_dhcp_opt.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.992068 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.992204 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: extra_dhcp_opt
Apr 21 13:29:25.992291 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: extraroute.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.992465 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.992600 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: extraroute
Apr 21 13:29:25.992654 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: extraroute_atomic.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.992900 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.993028 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: extraroute-atomic
Apr 21 13:29:25.993094 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: filter_validation.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.993346 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.993484 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: filter-validation
Apr 21 13:29:25.993548 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: fip_pf_description.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.993762 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.993919 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins
Apr 21 13:29:25.993990 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: fip_pf_detail.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.994181 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.994340 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: floating-ip-port-forwarding-detail
Apr 21 13:29:25.994411 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: fip_pf_port_range.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.994591 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.994727 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins
Apr 21 13:29:25.994800 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: fip_port_details.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.995004 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.995138 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: fip-port-details
Apr 21 13:29:25.995227 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: flavors.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.995572 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.995708 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: flavors
Apr 21 13:29:25.995813 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.996046 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.996184 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins
Apr 21 13:29:25.996267 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: floatingip_pools.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.996502 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.996635 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: floatingip-pools
Apr 21 13:29:25.996708 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: ip_allocation.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.996927 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.997093 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension ip_allocation not supported by any of loaded plugins
Apr 21 13:29:25.997161 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: ip_substring_port_filtering.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.997374 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.997551 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension ip-substring-filtering not supported by any of loaded plugins
Apr 21 13:29:25.997634 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l2_adjacency.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.997823 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.997959 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension l2_adjacency not supported by any of loaded plugins
Apr 21 13:29:25.998028 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.998465 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.998608 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: router
Apr 21 13:29:25.998673 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3_conntrack_helper.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.999300 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.999457 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension l3-conntrack-helper not supported by any of loaded plugins
Apr 21 13:29:25.999523 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:25.999727 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:25.999892 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: enable-default-route-bfd
Apr 21 13:29:25.999926 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.000118 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.000268 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: enable-default-route-ecmp
Apr 21 13:29:26.000338 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3_ext_gw_mode.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.000519 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.000656 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: ext-gw-mode
Apr 21 13:29:26.000725 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3_ext_ha_mode.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.000945 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.001073 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: l3-ha
Apr 21 13:29:26.001262 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.001333 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.001463 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins
Apr 21 13:29:26.001530 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3_extra_gws.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.001706 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.001839 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: external-gateway-multihoming
Apr 21 13:29:26.001908 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3_flavors.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.002082 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.002216 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: l3-flavors
Apr 21 13:29:26.002299 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3_ndp_proxy.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.002535 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.002669 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension l3-ndp-proxy not supported by any of loaded plugins
Apr 21 13:29:26.002737 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.002919 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.003048 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins
Apr 21 13:29:26.003118 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: l3agentscheduler.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.003579 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.003713 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension l3_agent_scheduler not supported by any of loaded plugins
Apr 21 13:29:26.003785 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: local_ip.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.004036 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.004176 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension local_ip not supported by any of loaded plugins
Apr 21 13:29:26.004256 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: logging.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.004501 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.004635 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension logging not supported by any of loaded plugins
Apr 21 13:29:26.004700 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: metering.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.005811 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.005956 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension metering not supported by any of loaded plugins
Apr 21 13:29:26.006006 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.006225 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.006370 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins
Apr 21 13:29:26.006439 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: multiprovidernet.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.006628 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.006758 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: multi-provider
Apr 21 13:29:26.006826 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: netmtu.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.007011 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.007139 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: net-mtu
Apr 21 13:29:26.007219 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: netmtu_writable.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.007399 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.007538 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: net-mtu-writable
Apr 21 13:29:26.007607 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: network_availability_zone.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.007806 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.007942 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: network_availability_zone
Apr 21 13:29:26.008011 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: network_ha.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.008190 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.008346 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension network_ha not supported by any of loaded plugins
Apr 21 13:29:26.008415 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: network_ip_availability.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.008608 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.008743 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: network-ip-availability
Apr 21 13:29:26.008814 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: network_ip_availability_details.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.009052 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.009188 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: network-ip-availability-details
Apr 21 13:29:26.009274 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: network_segment_range.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.009529 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.009656 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension network-segment-range not supported by any of loaded plugins
Apr 21 13:29:26.009724 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: pagination.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.009908 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.009981 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: pagination
Apr 21 13:29:26.010049 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: port_device_profile.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.010225 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.010375 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: port-device-profile
Apr 21 13:29:26.010446 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: port_hardware_offload_type.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.010619 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.010752 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension port-hardware-offload-type not supported by any of loaded plugins
Apr 21 13:29:26.010822 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.010995 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.011128 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins
Apr 21 13:29:26.011199 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: port_hints.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.011393 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.011526 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension port-hints not supported by any of loaded plugins
Apr 21 13:29:26.011594 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: port_mac_address_override.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.011769 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.011902 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension port-mac-override not supported by any of loaded plugins
Apr 21 13:29:26.011971 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: port_mac_address_regenerate.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.012144 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.012295 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: port-mac-address-regenerate
Apr 21 13:29:26.012359 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: port_numa_affinity_policy.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.012537 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.012669 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: port-numa-affinity-policy
Apr 21 13:29:26.012738 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.012933 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.013066 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: port-numa-affinity-policy-socket
Apr 21 13:29:26.013134 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: port_resource_request.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.013328 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.013460 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension port-resource-request not supported by any of loaded plugins
Apr 21 13:29:26.013529 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: port_resource_request_groups.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.013711 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.013842 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension port-resource-request-groups not supported by any of loaded plugins
Apr 21 13:29:26.013911 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: port_trusted_vif.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.014087 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.014224 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension port-trusted-vif not supported by any of loaded plugins
Apr 21 13:29:26.014304 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: portbindings.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.014482 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.014618 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: binding
Apr 21 13:29:26.014677 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: portbindings_extended.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.014891 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.015028 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: binding-extended
Apr 21 13:29:26.015093 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: portsecurity.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.015280 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.015414 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: port-security
Apr 21 13:29:26.015485 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: project_id.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.015669 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.015748 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: project-id
Apr 21 13:29:26.015802 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: providernet.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.015972 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.016103 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: provider
Apr 21 13:29:26.016165 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qinq.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.016374 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x8a88 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.016511 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: qinq
Apr 21 13:29:26.016566 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.017307 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.017449 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos not supported by any of loaded plugins
Apr 21 13:29:26.017508 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_bw_limit_direction.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.017730 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.017867 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-bw-limit-direction not supported by any of loaded plugins
Apr 21 13:29:26.017931 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.018118 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.018270 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins
Apr 21 13:29:26.018326 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_default.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.018772 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.018772 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-default not supported by any of loaded plugins
Apr 21 13:29:26.018772 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_fip.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.018889 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.019007 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-fip not supported by any of loaded plugins
Apr 21 13:29:26.019077 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_fip_network_policy.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.019274 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.019438 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-fip-network-policy not supported by any of loaded plugins
Apr 21 13:29:26.019509 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_gateway_ip.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.019690 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.019825 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-gateway-ip not supported by any of loaded plugins
Apr 21 13:29:26.019894 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_port_network_policy.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.020069 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.020201 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-port-network-policy not supported by any of loaded plugins
Apr 21 13:29:26.020289 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.020477 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.020612 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-pps-minimum not supported by any of loaded plugins
Apr 21 13:29:26.020681 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.020862 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.021024 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins
Apr 21 13:29:26.021093 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_pps_rule.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.021301 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.021432 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-pps not supported by any of loaded plugins
Apr 21 13:29:26.021501 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_rule_type_details.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.021677 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.021811 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-rule-type-details not supported by any of loaded plugins
Apr 21 13:29:26.021881 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_rule_type_filter.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.022054 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.022186 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-rule-type-filter not supported by any of loaded plugins
Apr 21 13:29:26.022274 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: qos_rules_alias.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.022463 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.022599 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension qos-rules-alias not supported by any of loaded plugins
Apr 21 13:29:26.022668 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: quota_check_limit.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.022848 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.022981 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: quota-check-limit
Apr 21 13:29:26.023049 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: quota_check_limit_default.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.023262 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.023394 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: quota-check-limit-default
Apr 21 13:29:26.023464 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: quotasv2.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.023878 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.024108 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: quotas
Apr 21 13:29:26.024180 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: quotasv2_detail.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.024480 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.024532 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: quota_details
Apr 21 13:29:26.024618 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: rbac.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.025053 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control project access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.025199 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: rbac-policies
Apr 21 13:29:26.025278 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: rbac_address_group.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.025483 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.025604 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension rbac-address-group not supported by any of loaded plugins
Apr 21 13:29:26.025663 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: rbac_address_scope.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.025855 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.025984 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: rbac-address-scope
Apr 21 13:29:26.026042 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: rbac_security_groups.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.026237 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.026371 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: rbac-security-groups
Apr 21 13:29:26.026429 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: rbac_subnetpool.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.026636 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.026776 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension rbac-subnetpool not supported by any of loaded plugins
Apr 21 13:29:26.026851 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: revisionifmatch.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.027032 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.027166 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: revision-if-match
Apr 21 13:29:26.027257 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: revisions.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.027451 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.027586 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: standard-attr-revisions
Apr 21 13:29:26.027644 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: router_availability_zone.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.027848 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.027983 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: router_availability_zone
Apr 21 13:29:26.028079 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: router_enable_snat.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.028248 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.028381 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: router-enable-snat
Apr 21 13:29:26.028451 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: routerservicetype.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.028626 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.028758 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension router-service-type not supported by any of loaded plugins
Apr 21 13:29:26.028827 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: security_groups_default_rules.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.029190 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.029343 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: security-groups-default-rules
Apr 21 13:29:26.029399 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.029598 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.029734 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: security-groups-normalized-cidr
Apr 21 13:29:26.029796 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: security_groups_port_filtering.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.029977 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.030102 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension port-security-groups-filtering not supported by any of loaded plugins
Apr 21 13:29:26.030160 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: security_groups_remote_address_group.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.030372 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.030498 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: security-groups-remote-address-group
Apr 21 13:29:26.030552 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.030740 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.030873 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: security-groups-rules-belongs-to-default-sg
Apr 21 13:29:26.030940 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: security_groups_shared_filtering.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.031108 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.031260 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: security-groups-shared-filtering
Apr 21 13:29:26.031324 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: securitygroup.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.032689 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.032849 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: security-group
Apr 21 13:29:26.032893 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: segment.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.033196 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.033351 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension segment not supported by any of loaded plugins
Apr 21 13:29:26.033427 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.033604 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.033742 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins
Apr 21 13:29:26.033810 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: servicetype.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.034019 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.034172 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: service-type
Apr 21 13:29:26.034259 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: sorting.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.034456 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.034519 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: sorting
Apr 21 13:29:26.034586 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: standard_attr_segment.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.034759 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.034898 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension standard-attr-segment not supported by any of loaded plugins
Apr 21 13:29:26.034951 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: standardattrdescription.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.035154 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.035305 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: standard-attr-description
Apr 21 13:29:26.035375 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: stateful_security_group.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.035552 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.035685 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: stateful-security-group
Apr 21 13:29:26.035749 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: stdattrs_common.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.035913 np0000168688 neutron-ovn-maintenance-worker[105483]: WARNING neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py
Apr 21 13:29:26.035969 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.036155 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.036310 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: subnet-dns-publish-fixed-ip
Apr 21 13:29:26.036382 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: subnet_external_network.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.036548 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.036681 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: subnet-external-network
Apr 21 13:29:26.036749 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: subnet_onboard.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.036937 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.037223 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension subnet_onboard not supported by any of loaded plugins
Apr 21 13:29:26.037310 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: subnet_segmentid_writable.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.037470 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.037753 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension subnet-segmentid-writable not supported by any of loaded plugins
Apr 21 13:29:26.037815 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: subnet_service_types.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.037997 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.038129 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: subnet-service-types
Apr 21 13:29:26.038187 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: subnetallocation.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.038405 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.038531 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: subnet_allocation
Apr 21 13:29:26.038765 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.038951 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.039075 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: subnetpool-prefix-ops
Apr 21 13:29:26.039143 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: tag_creation.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.039349 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.039477 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: tag-creation
Apr 21 13:29:26.039545 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.039723 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.039855 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins
Apr 21 13:29:26.039924 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: tagging.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.040644 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.040775 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: standard-attr-tag
Apr 21 13:29:26.040832 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: timestamp.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.041104 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.041230 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: standard-attr-timestamp
Apr 21 13:29:26.041314 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: trunk.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.041481 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.041611 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension trunk not supported by any of loaded plugins
Apr 21 13:29:26.041674 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: trunk_details.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.041854 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.041983 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension trunk-details not supported by any of loaded plugins
Apr 21 13:29:26.042052 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: uplink_status_propagation.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.042252 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.042381 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension uplink-status-propagation not supported by any of loaded plugins
Apr 21 13:29:26.042452 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.042635 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.042769 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins
Apr 21 13:29:26.042839 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loading extension file: vlantransparent.py {{(pid=105483) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 21 13:29:26.043032 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=105483) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 21 13:29:26.043169 np0000168688 neutron-ovn-maintenance-worker[105483]: INFO neutron.api.extensions [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Loaded extension: vlan-transparent
Apr 21 13:29:26.045390 np0000168688 neutron-ovn-maintenance-worker[105483]: WARNING oslo_service.backend._threading.service [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback.
Apr 21 13:29:26.045666 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Full set of CONF: {{(pid=105483) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}}
Apr 21 13:29:26.045731 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ******************************************************************************** {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
Apr 21 13:29:26.045793 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Configuration options gathered from: {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
Apr 21 13:29:26.045865 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
Apr 21 13:29:26.045963 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
Apr 21 13:29:26.046017 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ================================================================================ {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
Apr 21 13:29:26.046139 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] agent_down_time                = 75 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.046233 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] allow_automatic_dhcp_failover  = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.046326 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] allow_bulk                     = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.046491 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.046576 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] api_extensions_path            =  {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.046665 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] api_paste_config               = api-paste.ini {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.046756 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] api_workers                    = 4 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.046842 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] auth_strategy                  = keystone {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.046930 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] backlog                        = 4096 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.047015 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] base_mac                       = fa:16:3e:00:00:00 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.047118 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] bind_host                      = 0.0.0.0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.047226 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] bind_port                      = 9696 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.047310 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] client_socket_timeout          = 900 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.047402 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] config_dir                     = [] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.047469 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.047556 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] config_source                  = [] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.047649 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] control_exchange               = neutron {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.047737 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] core_plugin                    = ml2 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.047824 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] debug                          = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.047910 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] default_availability_zones     = [] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.048054 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.048147 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] dhcp_agent_notification        = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.048246 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] dhcp_agents_per_network        = 1 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.048332 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] dhcp_lease_duration            = 86400 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.048421 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] dhcp_load_type                 = networks {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.048508 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] dns_domain                     = openstacklocal {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.048596 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.048680 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] enable_default_route_bfd       = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.048763 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] enable_default_route_ecmp      = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.048847 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] enable_dvr                     = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.048931 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] enable_new_agents              = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.049015 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] enable_services_on_agents_with_admin_state_down = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.049142 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] enable_snat_by_default         = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.049220 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] enable_traditional_dhcp        = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.049317 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] external_dns_driver            = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.049399 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] filter_validation              = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.049492 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] global_physnet_mtu             = 1430 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.049544 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] graceful_shutdown_timeout      = 60 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.049671 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] host                           = np0000168688 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.049759 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] host_dvr_for_dhcp              = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.049867 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] http_retries                   = 3 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.049959 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] instance_format                = [instance: %(uuid)s]  {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050045 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050131 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ipam_driver                    = internal {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050231 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ipv6_pd_enabled                = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050322 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] l3_ha                          = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050403 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050490 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] l3_ha_network_physical_name    =  {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050579 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] l3_ha_network_type             =  {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050652 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] log_color                      = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050750 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] log_config_append              = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050838 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050930 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] log_dir                        = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.050987 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] log_file                       = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051060 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] log_options                    = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051143 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] log_rotate_interval            = 1 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051247 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] log_rotate_interval_type       = days {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051337 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] log_rotation_type              = none {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051400 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051465 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051553 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051638 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051701 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051785 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] max_allowed_address_pair       = 10 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051869 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] max_dns_nameservers            = 5 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.051952 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] max_header_line                = 16384 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.052036 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] max_l3_agents_per_router       = 3 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.052144 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] max_logfile_count              = 30 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.052256 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] max_logfile_size_mb            = 200 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.052341 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] max_routes                     = 30 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.052424 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] max_subnet_host_routes         = 20 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.052511 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] my_ip                          = 199.19.213.86 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.052601 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] my_ipv6                        = 2604:e100:1:0:f816:3eff:fef2:3dda {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.052680 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] network_auto_schedule          = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.052759 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] network_link_prefix            = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.052843 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.052929 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] notify_nova_on_port_data_changes = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053013 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] notify_nova_on_port_status_changes = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053100 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] pagination_max_limit           = -1 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053202 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] periodic_fuzzy_delay           = 5 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053302 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] periodic_interval              = 40 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053381 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] publish_errors                 = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053463 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] rate_limit_burst               = 0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053551 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] rate_limit_except_level        = CRITICAL {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053634 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] rate_limit_interval            = 0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053723 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] retry_until_window             = 30 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053790 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] router_distributed             = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053883 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] rpc_resources_processing_step  = 20 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.053965 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] rpc_response_max_timeout       = 600 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.054049 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] rpc_response_timeout           = 60 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.054137 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] rpc_state_report_workers       = 0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.054221 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] rpc_workers                    = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.054319 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] send_events_interval           = 2 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.054410 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] service_plugins                = ['ovn-router'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.054497 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] setproctitle                   = on {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.054577 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] shell_completion               = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.054662 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] state_path                     = /opt/stack/data/neutron {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.054750 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] syslog_log_facility            = LOG_USER {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.054834 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] tcp_keepidle                   = 600 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.054927 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] transport_url                  = **** {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055010 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] use_journal                    = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055075 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] use_json                       = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055166 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] use_ssl                        = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055265 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] use_stderr                     = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055353 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] use_syslog                     = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055435 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] vlan_qinq                      = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055515 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] vlan_transparent               = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055597 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] wsgi_default_pool_size         = 100 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055680 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] wsgi_keep_alive                = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055767 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055850 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] wsgi_server_debug              = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.055948 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_concurrency.disable_process_locking = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.056043 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.056146 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler.connection_string     = messaging:// {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.056249 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler.enabled               = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.056341 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler.es_doc_type           = notification {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.056427 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler.es_scroll_size        = 10000 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.056520 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler.es_scroll_time        = 2m {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.056605 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler.filter_error_trace    = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.056693 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler.hmac_keys             = **** {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.056782 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler.sentinel_service_name = mymaster {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.056876 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler.socket_timeout        = 0.1 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.056961 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler.trace_requests        = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.057045 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler.trace_sqlalchemy      = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.057139 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler_jaeger.process_tags   = {} {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.057267 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler_jaeger.service_name_prefix = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.057366 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] profiler_otlp.service_name_prefix = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.057452 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.057533 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.057619 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.057708 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.057793 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.057887 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_policy.enforce_new_defaults = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.058063 np0000168688 neutron-ovn-maintenance-worker[105483]: WARNING oslo_config.cfg [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.).  Its value may be silently ignored in the future.
Apr 21 13:29:26.058151 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_policy.enforce_scope      = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.058258 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_policy.policy_default_rule = default {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.058352 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.058441 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.058531 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.058616 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.058698 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.058807 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.058908 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.058994 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_policy.remote_timeout     = 60.0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.059088 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.059175 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] service_providers.service_provider = [] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.059292 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.auth_section              = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.059385 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.auth_type                 = password {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.059475 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.auth_url                  = https://199.19.213.86/identity {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.059564 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.cafile                    = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.059646 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.certfile                  = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.059734 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.collect_timing            = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.059961 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.default_domain_id         = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.059961 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.default_domain_name       = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.059961 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.domain_id                 = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060255 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.domain_name               = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060255 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.endpoint_type             = public {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060255 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.insecure                  = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060353 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.keyfile                   = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060388 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.password                  = **** {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060479 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.project_domain_id         = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060592 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.project_domain_name       = Default {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060651 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.project_id                = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060742 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.project_name              = service {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060869 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.region_name               = RegionOne {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060917 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.split_loggers             = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.060992 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.system_scope              = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.061075 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.timeout                   = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.061158 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.trust_id                  = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.061275 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.user_domain_id            = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.061364 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.user_domain_name          = Default {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.061448 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.user_id                   = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.061536 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] nova.username                  = nova {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.061626 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] placement.auth_section         = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.061735 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] placement.auth_type            = password {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.061802 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] placement.cafile               = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.061883 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] placement.certfile             = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.061969 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] placement.collect_timing       = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062058 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] placement.endpoint_type        = public {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062142 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] placement.insecure             = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062240 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] placement.keyfile              = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062325 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] placement.region_name          = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062410 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] placement.split_loggers        = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062491 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] placement.timeout              = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062580 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.auth_section            = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062665 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.auth_type               = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062746 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.cafile                  = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062828 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.certfile                = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062920 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.collect_timing          = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.062997 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.connect_retries         = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.063080 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.connect_retry_delay     = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.063163 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.enable_notifications    = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.063265 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.endpoint_override       = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.063350 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.insecure                = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.063432 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.interface               = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.063513 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.keyfile                 = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.063595 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.max_version             = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.063678 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.min_version             = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.063760 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.region_name             = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.063843 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.retriable_status_codes  = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.063925 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.service_name            = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064007 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.service_type            = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064093 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.split_loggers           = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064174 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.status_code_retries     = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064272 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.status_code_retry_delay = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064357 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.timeout                 = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064441 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.valid_interfaces        = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064524 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ironic.version                 = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064612 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] cli_script.dry_run             = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064707 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064791 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064876 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.064961 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.065045 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.065128 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.065260 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.065346 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.065433 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.065523 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.hostname = np0000168688 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.065604 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.065694 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.065822 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.065869 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.065948 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066035 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066119 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066224 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066313 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066398 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066484 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066568 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066654 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066738 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066824 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066909 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.066994 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.067107 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.067195 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.067299 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.067384 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.ssl      = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.067474 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.067561 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.067646 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.067733 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.067820 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.067904 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.067993 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_notifications.driver = [] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.068079 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_notifications.retry = -1 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.068172 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.068302 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_messaging_notifications.transport_url = **** {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.068406 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_reports.file_event_handler = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.068494 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.068578 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_reports.log_dir           = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.068667 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.068757 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QUOTAS.default_quota           = -1 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.068846 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.068931 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QUOTAS.quota_floatingip        = 50 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.069016 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QUOTAS.quota_network           = 100 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.069103 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QUOTAS.quota_port              = 500 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.069189 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QUOTAS.quota_rbac_policy       = 10 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.069317 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QUOTAS.quota_router            = 10 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.069404 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QUOTAS.quota_security_group    = 10 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.069489 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QUOTAS.quota_security_group_rule = 100 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.069573 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QUOTAS.quota_subnet            = 100 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.069659 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] QUOTAS.track_quota_usage       = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.069753 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.069838 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep.group                  = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.069921 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep.helper_command         = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070007 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep.log_daemon_traceback   = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070096 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep.logger_name            = oslo_privsep.daemon {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070181 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep.thread_pool_size       = 16 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070281 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep.user                   = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070372 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070457 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_dhcp_release.group     = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070536 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_dhcp_release.helper_command = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070622 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070709 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070794 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_dhcp_release.thread_pool_size = 16 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070880 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_dhcp_release.user      = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.070966 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071048 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_ovs_vsctl.group        = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071130 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_ovs_vsctl.helper_command = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071217 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071317 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071416 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071484 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_ovs_vsctl.user         = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071574 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_namespace.capabilities = [21] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071658 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_namespace.group        = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071740 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_namespace.helper_command = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071825 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_namespace.log_daemon_traceback = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071912 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.071997 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_namespace.thread_pool_size = 16 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.072080 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_namespace.user         = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.072169 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_conntrack.capabilities = [12] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.072267 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_conntrack.group        = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.072350 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_conntrack.helper_command = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.072438 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_conntrack.log_daemon_traceback = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.072521 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.072608 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_conntrack.thread_pool_size = 16 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.072691 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_conntrack.user         = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.072781 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_link.capabilities      = [12, 21] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.072863 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_link.group             = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.072944 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_link.helper_command    = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.073027 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_link.log_daemon_traceback = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.073114 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.073199 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_link.thread_pool_size  = 16 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.073340 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] privsep_link.user              = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.073426 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.073517 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.073603 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] AGENT.use_helper_for_ns_read   = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.073693 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] SECURITYGROUP.enable_ipset     = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.073779 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] SECURITYGROUP.enable_security_group = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.073862 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] SECURITYGROUP.firewall_driver  = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.073948 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.074037 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] experimental.ipv6_pd_enabled   = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.074137 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.074228 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2.external_network_type      = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.074330 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2.mechanism_drivers          = ['ovn'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.074420 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2.overlay_ip_version         = 4 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.074506 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2.path_mtu                   = 0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.074595 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2.physical_network_mtus      = {} {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.074691 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2.project_network_types      = ['geneve'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.074780 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.074874 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.074967 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2_type_flat.flat_networks    = ['public'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.075064 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2_type_geneve.max_header_size = 38 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.075156 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.075263 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.075356 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] network_log.burst_limit        = 25 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.075442 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] network_log.local_output_log_base = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.075529 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] network_log.rate_limit         = 100 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.075618 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.asyncio_connection    = **** {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.075701 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.asyncio_slave_connection = **** {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.075791 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.backend               = sqlalchemy {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.075880 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.connection            = **** {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.075967 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.connection_debug      = 0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.076056 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.connection_parameters =  {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.076142 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.connection_recycle_time = 3600 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.076244 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.connection_trace      = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.076330 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.db_inc_retry_interval = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.076416 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.db_max_retries        = 20 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.076505 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.db_max_retry_interval = 10 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.076590 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.db_retry_interval     = 1 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.076675 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.max_overflow          = 50 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.076762 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.max_pool_size         = 5 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.076847 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.max_retries           = 10 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.076936 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077020 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.mysql_wsrep_sync_wait = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077104 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.pool_timeout          = None {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077188 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.retry_interval        = 10 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077318 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.slave_connection      = **** {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077399 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.sqlite_synchronous    = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077480 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.synchronous_reader    = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077565 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] database.use_db_reconnect      = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077658 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.bfd_min_rx                 = 1000 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077745 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.bfd_min_tx                 = 100 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077828 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.bfd_mult                   = 3 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077912 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.broadcast_arps_to_all_routers = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.077997 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.dhcp_default_lease_time    = 43200 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.078083 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.078167 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.dns_records_ovn_owned      = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.078271 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.dns_servers                = [] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.078361 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.enable_distributed_floating_ip = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.078444 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.fdb_age_threshold          = 300 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.078533 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ha_failover_strategy       = normal {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.078622 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.live_migration_activation_strategy = rarp {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.078707 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.localnet_learn_fdb         = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.078793 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.mac_binding_age_threshold  = 300 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.078885 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.neutron_sync_mode          = log {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.078968 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_dhcp4_global_options   = {} {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.079052 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_dhcp6_global_options   = {} {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.079142 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_l3_scheduler           = leastloaded {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.079246 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_metadata_enabled       = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.079336 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.079423 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.079515 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_nb_connection          = ['ssl:199.19.213.86:6641'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.079602 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.079687 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_router_indirect_snat   = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.079773 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.079858 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.079948 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_sb_connection          = ['ssl:199.19.213.86:6642'] {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.080035 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.080120 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovs_create_tap             = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.080204 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovsdb_connection_timeout   = 180 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.080311 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovsdb_log_level            = INFO {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.080397 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovsdb_probe_interval       = 60000 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.080480 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.ovsdb_retry_max_interval   = 180 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.080564 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.stateless_nat_enabled      = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.080654 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.080745 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] OVS.bridge_mac_table_size      = 50000 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.080831 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] OVS.igmp_flood                 = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.080917 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] OVS.igmp_flood_reports         = True {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.081003 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] OVS.igmp_flood_unregistered    = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.081092 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] OVS.igmp_snooping_enable       = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.081179 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] OVS.ovsdb_timeout              = 10 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.081286 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.081407 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn_nb_global.ignore_lsp_down  = False {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.081496 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.081548 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG cotyledon.oslo_config_glue [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] ******************************************************************************** {{(pid=105483) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
Apr 21 13:29:26.081641 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG oslo_service.backend._threading.service [None req-b999acb7-54fe-464f-90c8-233dc9b41123 None None] Selected the multiprocessing context: <multiprocessing.context.ForkContext object at 0x7d4d588669f0> and updated it in Cotyledon ServiceManager: <multiprocessing.context.ForkContext object at 0x7d4d588669f0> {{(pid=105483) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:394}}
Apr 21 13:29:26.113688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=106267) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}}
Apr 21 13:29:26.114242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
Apr 21 13:29:26.114242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
Apr 21 13:29:26.114242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
Apr 21 13:29:26.114401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
Apr 21 13:29:26.114401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
Apr 21 13:29:26.114635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time                = 75 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.114707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover  = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.114792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk                     = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.114961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.115044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path            =  {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.115094 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config               = api-paste.ini {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.115195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] api_workers                    = 4 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.115308 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy                  = keystone {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.115396 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] backlog                        = 4096 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.115473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] base_mac                       = fa:16:3e:00:00:00 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.115642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] bind_host                      = 0.0.0.0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.115731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] bind_port                      = 9696 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.115809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout          = 900 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.115899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] config_dir                     = [] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.115947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] config_source                  = [] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] control_exchange               = neutron {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116203 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] core_plugin                    = ml2 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] debug                          = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones     = [] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification        = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network        = 1 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116779 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration            = 86400 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type                 = networks {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] dns_domain                     = openstacklocal {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.116993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.117055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd       = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.117132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp      = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.117227 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr                     = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.117315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents              = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.117410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.117482 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default         = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.117626 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp        = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.117691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver            = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.117869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations             = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.117869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] filter_validation              = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.117869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu             = 1430 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118007 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout      = 60 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] host                           = np0000168688 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118238 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp              = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] http_retries                   = 3 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] instance_format                = [instance: %(uuid)s]  {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver                    = internal {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled                = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118657 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] l3_ha                          = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118735 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name    =  {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118881 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type             =  {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.118951 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] log_color                      = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] log_config_append              = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119148 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] log_dir                        = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] log_file                       = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119277 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] log_options                    = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval            = 1 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type       = days {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type              = none {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119583 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair       = 10 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers            = 5 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.119923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] max_header_line                = 16384 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120007 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router       = 3 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count              = 30 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120137 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb            = 200 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] max_routes                     = 30 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120304 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes         = 20 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] my_ip                          = 199.19.213.86 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6                        = 2604:e100:1:0:f816:3eff:fef2:3dda {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule          = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix            = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit           = -1 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.120970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay           = 5 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval              = 40 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] publish_errors                 = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst               = 0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121261 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level        = CRITICAL {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval            = 0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] retry_until_window             = 30 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] router_distributed             = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121547 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step  = 20 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout       = 600 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121754 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout           = 60 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers       = 0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121907 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers                    = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.121983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval           = 2 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] service_plugins                = ['ovn-router'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] setproctitle                   = on {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] shell_completion               = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122328 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] state_path                     = /opt/stack/data/neutron {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility            = LOG_USER {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle                   = 600 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] transport_url                  = **** {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] use_journal                    = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] use_json                       = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] use_ssl                        = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122802 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] use_stderr                     = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122883 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] use_syslog                     = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.122942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq                      = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.123002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent               = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.123064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size         = 100 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.123126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive                = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.123204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.123286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug              = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 21 13:29:26.123417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.123611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string     = messaging:// {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.123694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled               = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.123770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type           = notification {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.123846 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size        = 10000 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.123909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time        = 2m {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.123988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace    = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys             = **** {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout        = 0.1 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests        = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124391 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy      = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags   = {} {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124609 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name =  {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.124931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125004 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125102 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope      = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125274 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125588 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout     = 60.0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125746 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section              = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type                 = password {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url                  = https://199.19.213.86/identity {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.125962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile                    = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile                  = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing            = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126102 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id         = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name       = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126180 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id                 = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name               = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type             = public {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure                  = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile                   = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.password                  = **** {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id         = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name       = Default {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id                = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name              = service {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name               = RegionOne {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers             = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope              = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout                   = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id                  = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126819 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id            = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name          = Default {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id                   = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126928 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] nova.username                  = nova {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.126993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section         = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127043 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type            = password {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile               = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile             = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing       = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127194 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type        = public {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure             = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127294 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile              = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name          = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers        = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout              = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section            = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type               = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile                  = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile                = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing          = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries         = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127703 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay     = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications    = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override       = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure                = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127873 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface               = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile                 = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127948 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version             = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.127981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version             = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128029 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name             = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes  = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name            = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type            = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers           = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries     = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout                 = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces        = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ironic.version                 = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run             = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128588 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128621 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.128934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129004 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000168688 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129108 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129238 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129278 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129547 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129744 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129824 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129870 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.129998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl      = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130038 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file =  {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130223 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version =  {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir           = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130726 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota           = -1 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip        = 50 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network           = 100 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.130972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port              = 500 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131022 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy       = 10 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router            = 10 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group    = 10 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131222 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet            = 100 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131300 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage       = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep.group                  = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command         = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131529 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback   = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name            = oslo_privsep.daemon {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size       = 16 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep.user                   = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group     = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 16 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.131963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user      = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group        = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user         = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group        = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132440 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 16 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132543 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user         = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group        = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132894 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 16 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user         = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.132974 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities      = [12, 21] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group             = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command    = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133200 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size  = 16 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user              = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133310 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read   = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset     = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133502 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver  = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled   = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type      = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers          = ['ovn'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version         = 4 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.133940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu                   = 0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus      = {} {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types      = ['geneve'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks    = ['public'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134350 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit        = 25 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit         = 100 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection    = **** {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.backend               = sqlalchemy {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.connection            = **** {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug      = 0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134808 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters =  {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace      = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.134979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries        = 20 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135020 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval     = 1 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow          = 50 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size         = 5 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135194 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries           = 10 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135270 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode        = TRADITIONAL {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout          = None {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval        = 10 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection      = **** {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135493 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous    = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135535 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader    = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135593 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect      = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx                 = 1000 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx                 = 100 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135766 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult                   = 3 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135808 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time    = 43200 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned      = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.135974 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers                = [] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold          = 300 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy       = normal {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb         = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold  = 300 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode          = log {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options   = {} {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136436 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options   = {} {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler           = leastloaded {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled       = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.136688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection          = ['ssl:199.19.213.86:6641'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137089 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat   = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137294 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection          = ['ssl:199.19.213.86:6642'] {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap             = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137504 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout   = 180 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137542 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level            = INFO {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137589 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval       = 60000 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval   = 180 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137668 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled      = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.137777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.138051 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size      = 50000 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.138128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood                 = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.138207 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports         = True {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.138265 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered    = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.138317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable       = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.138359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout              = 10 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.138407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.138446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down  = False {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.138498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 21 13:29:26.138545 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=106267) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
Apr 21 13:29:26.138865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [106267] {{(pid=106267) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}}
Apr 21 13:29:26.139822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-8251486', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-11548'] for process (None), after_init {{(pid=106267) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 21 13:29:26.141035 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry
Apr 21 13:29:26.195912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 21 13:29:26.196054 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 21 13:29:26.196319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 21 13:29:26.196463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.196561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.196662 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.196761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.196835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.196953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.197044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.197132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.197291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.197360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.197488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.197589 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.201682 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.86:6641: connecting...
Apr 21 13:29:26.207686 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.86:6641: connected
Apr 21 13:29:26.234893 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry
Apr 21 13:29:26.261191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 21 13:29:26.261310 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 21 13:29:26.261310 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.261411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=106267) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 21 13:29:26.263289 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.86:6642: connecting...
Apr 21 13:29:26.269509 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.86:6642: connected
Apr 21 13:29:26.302196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7d4d52b1dfa0>> floatingip precommit_update 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7d4d52b1dfa0>> floatingip precommit_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.302401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7d4d52b1dfa0>> floatingip before_create 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.302474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7d4d52b1dfa0>> floatingip before_update 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.302537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7d4d52b1dfa0>> port after_update 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.302626 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7d4d52b1dfa0>> port precommit_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.302877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-12237'] for port_forwarding_plugin (None), after_init {{(pid=106267) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 21 13:29:26.302941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7d4d5330d550>> port_forwarding after_create 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.302997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7d4d5330d550>> router_interface after_create 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.303072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7d4d5330d550>> port_forwarding after_update 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.303136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7d4d5330d550>> port_forwarding after_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.303194 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7d4d5330d550>> router_interface after_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.303344 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Apr 21 13:29:26.303426 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension
Apr 21 13:29:26.303777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._prevent_segment_delete_with_subnet_associated of <neutron.services.segments.plugin.Plugin object at 0x7d4d537e8920>> segment before_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.303864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._validate_auto_address_subnet_delete of <neutron.services.segments.plugin.Plugin object at 0x7d4d537e8920>> subnet precommit_delete_associations 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.304146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_addition_to_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7d4d53804440>> segment_host_mapping after_create 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.304211 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_removal_from_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7d4d53804440>> segment_host_mapping after_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.304294 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7d4d53804440>> port after_create 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.304382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7d4d53804440>> port after_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.304441 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7d4d53804440>> port after_update 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.304501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_segment_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7d4d53804440>> segment after_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.304562 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_created of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7d4d53804440>> subnet after_create 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.304618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7d4d53804440>> subnet after_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.304668 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7d4d53804440>> subnet after_update 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.362392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7d4d53823b90>> subnet after_create 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.362392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_delete of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7d4d53823b90>> subnet after_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.362675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7d4d53823b90>> subnet before_create 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.362675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_update of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7d4d53823b90>> subnet before_update 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.364798 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_resource_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7d4d537de630>> security_group after_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.364926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_target_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7d4d537de630>> port after_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.365008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-812698'] for logging-plugin (None), after_init {{(pid=106267) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 21 13:29:26.365093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7d4d537cb650>> security_group_rule after_create 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.365156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7d4d537cb650>> security_group_rule after_update 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:26.365256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7d4d537cb650>> security_group_rule after_delete 55550000, False {{(pid=106267) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 21 13:29:35.840242 np0000168688 neutron-ovn-maintenance-worker[105483]: DEBUG dbcounter [-] [105483] Writing DB stats neutron:SELECT=4 {{(pid=105483) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:29:36.366314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-04-21 13:29:36.365927 {{(pid=106267) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}}
Apr 21 13:29:36.369229 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync port groups started @ 2026-04-21 13:29:36.368908 {{(pid=106267) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}}
Apr 21 13:29:36.386761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writer thread running {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
Apr 21 13:29:36.401091 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync port groups completed @ 2026-04-21 13:29:36.400808 {{(pid=106267) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}}
Apr 21 13:29:36.401311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-04-21 13:29:36.401061 {{(pid=106267) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1150}}
Apr 21 13:29:36.516438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.services.segments.db [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (<neutron_lib.context.Context object at 0x7d4d53827200>,) {'filters': {'network_id': ['b57c058f-054e-4a2a-980f-6bfda2111e90'], 'is_dynamic': False}} {{(pid=106267) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
Apr 21 13:29:36.522066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync metadata ports started {{(pid=106267) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1095}}
Apr 21 13:29:36.747026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync metadata ports completed {{(pid=106267) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1147}}
Apr 21 13:29:36.747567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=106267) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:947}}
Apr 21 13:29:36.812692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-04-21 13:29:36.812416 {{(pid=106267) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1020}}
Apr 21 13:29:36.813027 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 21 13:29:36.813206 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=106267) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1025}}
Apr 21 13:29:36.813346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=106267) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1084}}
Apr 21 13:29:36.813441 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-04-21 13:29:36.813281 {{(pid=106267) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1319}}
Apr 21 13:29:36.813517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync ACLs started @ 2026-04-21 13:29:36.813404 {{(pid=106267) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}}
Apr 21 13:29:36.855753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync ACLs completed @ 2026-04-21 13:29:36.855457 {{(pid=106267) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:429}}
Apr 21 13:29:36.855911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync routers and router ports started @ 2026-04-21 13:29:36.855767 {{(pid=106267) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:734}}
Apr 21 13:29:36.908070 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 21 13:29:36.908416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync routers and router ports completed @ 2026-04-21 13:29:36.908116 {{(pid=106267) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:942}}
Apr 21 13:29:36.908582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync port QoS policies started @ 2026-04-21 13:29:36.908386 {{(pid=106267) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1367}}
Apr 21 13:29:36.908975 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] Starting OVNClientQosExtension
Apr 21 13:29:36.926053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync port QoS policies completed @ 2026-04-21 13:29:36.925744 {{(pid=106267) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1390}}
Apr 21 13:29:36.926053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync Floating IP QoS policies started @ 2026-04-21 13:29:36.925956 {{(pid=106267) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1395}}
Apr 21 13:29:36.926351 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] Starting OVNClientQosExtension
Apr 21 13:29:36.945435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-04-21 13:29:36.945199 {{(pid=106267) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1418}}
Apr 21 13:29:36.945635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync Floating IP NAT rules started @ 2026-04-21 13:29:36.945375 {{(pid=106267) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1423}}
Apr 21 13:29:36.945720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-04-21 13:29:36.945590 {{(pid=106267) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1446}}
Apr 21 13:29:36.945814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-a91df235-193e-4ffe-a705-b53330a8382a None None] OVN-Northbound DB sync process completed @ 2026-04-21 13:29:36.945681 {{(pid=106267) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}}
Apr 21 13:29:46.367854 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-04-21 13:29:46.367385 {{(pid=106267) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1474}}
Apr 21 13:29:46.368983 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Apr 21 13:29:46.373162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-412379ac-0333-47e4-9d03-fc1dfbaa1237 None None] OVN-SB Sync hostname and physical networks started @ 2026-04-21 13:29:46.372846 {{(pid=106267) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1501}}
Apr 21 13:29:46.375323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.377071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.377071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.377224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.377737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.378465 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.378620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.379037 np0000168688 neutron-ovn-maintenance-worker[106267]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-412379ac-0333-47e4-9d03-fc1dfbaa1237 None None] New host np0000168688 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron
Apr 21 13:29:46.379306 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.379444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.379554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.379837 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.380091 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.380243 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.381190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.381361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.383954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.384048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.384088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.384138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.384245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.384366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.384769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.386081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_static_routes {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.386151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.386429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_virtual_port_parent_hostname {{(pid=106267) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 21 13:29:46.389996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.391321 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started
Apr 21 13:29:46.391417 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization
Apr 21 13:29:46.393065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG oslo_concurrency.lockutils [None req-412379ac-0333-47e4-9d03-fc1dfbaa1237 None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=106267) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Apr 21 13:29:46.394248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.395000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG oslo_concurrency.lockutils [None req-412379ac-0333-47e4-9d03-fc1dfbaa1237 None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.001s {{(pid=106267) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Apr 21 13:29:46.395556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_baremetal_ports_dhcp_options {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.396565 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Sync baremetal port DHCP options with configuration. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.400337 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.401635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_dhcp_options_consistency {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.401994 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Backfill network_id on DHCP_Options external_ids. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.405235 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.413343 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fair_meter_consistency {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.420956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.422623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Refresh ACL logging fair meter after configuration reload. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.423747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.429380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron_lib.callbacks.manager [None req-412379ac-0333-47e4-9d03-fc1dfbaa1237 None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-1910241', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-7904310'] for segment_host_mapping (None), after_create {{(pid=106267) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 21 13:29:46.437187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.457024 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_fair_meter_consistency finished in 0.000 seconds
Apr 21 13:29:46.458828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_igmp_snoop_support {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.460230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.469931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.472075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.474254 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:46.477321 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Check for IGMP snoop support. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.478353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.479196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.480748 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_global_dhcp_opts {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.491403 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_for_igmp_snoop_support finished in 0.001 seconds
Apr 21 13:29:46.492483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:29:46.503590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.504706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Check global DHCP options consistency. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.536630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.services.segments.db [None req-412379ac-0333-47e4-9d03-fc1dfbaa1237 None None] Segments {'19f04b40-ed99-482a-8774-a76892a716d1'} mapped to the host np0000168688 {{(pid=106267) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}}
Apr 21 13:29:46.540393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_localnet_port_has_learn_fdb {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.547601 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:46.547808 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.551644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_network_broadcast_arps_to_all_routers {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.552412 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_global_dhcp_opts finished in 0.000 seconds
Apr 21 13:29:46.558359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Ensure localnet ports have learn-fdb set per configuration. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.560053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Sync broadcast-arps-to-all-routers on external networks. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.562519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.575147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_provider_distributed_ports {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.586419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:46.591174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG oslo_concurrency.lockutils [None req-412379ac-0333-47e4-9d03-fc1dfbaa1237 None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.167s {{(pid=106267) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Apr 21 13:29:46.592065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 21 13:29:46.600087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.613461 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Align reside-on-redirect-chassis on VLAN/FLAT distributed ports. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.613726 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task check_dhcp_options_consistency finished in 0.172 seconds
Apr 21 13:29:46.616888 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_redirect_type_router_gateway_ports {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.627838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-412379ac-0333-47e4-9d03-fc1dfbaa1237 None None] OVN-SB Sync hostname and physical networks completed @ 2026-04-21 13:29:46.590801 {{(pid=106267) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1528}}
Apr 21 13:29:46.628657 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e73d9861-2bd0-4385-bbc9-43af34634e7a None None] Maintenance task: No inconsistencies found. Skipping {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:436}}
Apr 21 13:29:46.638742 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_localnet_port_has_learn_fdb finished in 0.051 seconds
Apr 21 13:29:46.649853 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-87bf22d3-15db-443a-bf41-2189f2f1c4f0 None None] OVN maintenance task check_baremetal_ports_dhcp_options finished in 0.215 seconds
Apr 21 13:29:46.650943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.654097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:46.654145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.660900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Align router gateway redirect-type for provider VLAN/FLAT networks. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.676631 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e73d9861-2bd0-4385-bbc9-43af34634e7a None None] OVN maintenance task check_for_inconsistencies finished in 0.151 seconds
Apr 21 13:29:46.684321 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:46.685639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [None req-87bf22d3-15db-443a-bf41-2189f2f1c4f0 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:46.685639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.686569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: cleanup_old_hash_ring_nodes {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.700285 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.709290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Purge stale hash ring nodes older than five days. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.712319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:46.724676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.725866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: configure_nb_global {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.732414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.742168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply ovn_nb_global settings to NB_Global options. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.746480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.747187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:29:46.747564 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 21 13:29:46.747956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:46.754035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:46.754035 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:46.754035 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:46.754035 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:46.754035 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:46.754035 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:46.754035 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.754035 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.069 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:46.787886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:46.788404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:46.788803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.788803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:46.791925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:46.796867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.802805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.803635 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.db.ovn_hash_ring_db [None req-6d435a63-28f9-4611-a040-3911dc941ea8 None None] Cleaned up Hash Ring nodes older than 5 days
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:46.817702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:46.818689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:46.819884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.819884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:46.820634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.823393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.828499 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6d435a63-28f9-4611-a040-3911dc941ea8 None None] OVN maintenance task cleanup_old_hash_ring_nodes finished in 0.094 seconds
Apr 21 13:29:46.832280 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.833930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_network_type_and_physnet {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.848374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:46.848374 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:46.848374 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:46.848374 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:46.848374 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:46.848374 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:46.848374 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.848374 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:46.869554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:29:46.888965 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set network type and physnet external_ids on logical switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.907168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:29:46.923736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:46.924632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:29:46.924632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:46.924632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:29:46.924632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:29:46.924632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:46.924632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:29:46.924632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.924632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:29:46.930183 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_ovn_owned_dns_option {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.932929 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:46.942552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.943201 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.943414 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.943758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:29:46.959746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task configure_nb_global failed after 0.186 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 990, in configure_nb_global
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:46.971661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:46.972245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:46.972659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:46.972659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.972659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:46.974524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply ovn_owned option on DNS records per configuration. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:46.975985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:46.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_lrouter_ports_ext_ids_name_prefix {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:46.985559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:46.985559 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:46.985559 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:46.985559 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:46.985559 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:46.985559 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:46.985559 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.985559 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:46.988748 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-3b6cf2fd-c3f9-4a73-8c3d-452ab7c6f428 None None] OVN maintenance task check_network_broadcast_arps_to_all_routers finished in 0.410 seconds
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:46.995771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 990, in configure_nb_global
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:46.996603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:46.997553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:46.997553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:46.997553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:46.997553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:47.009044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:47.015999 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: ('Check missing prefix router_name', 'in external_ids of LRPs.') {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:47.019100 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:29:47.032430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:47.032954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:29:47.032954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.032954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:29:47.032954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.032954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.032954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:29:47.032954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.032954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:29:47.033185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [None req-3b6cf2fd-c3f9-4a73-8c3d-452ab7c6f428 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:47.043115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:47.043403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:47.046334 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.209 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.071065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:47.071570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.071969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.071969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:47.071969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.071969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:47.073404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_router_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:47.090206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.094734 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:29:47.100538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:47.107699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_router_static_routes {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.115232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:47.115864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.116473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.116473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:47.116473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.116473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:47.116473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Sync router.distributed with distributed floating IP config. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:47.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_ovn_owned_dns_option finished in 0.120 seconds
Apr 21 13:29:47.118473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:47.118473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.118473 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:47.118473 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.118473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.118473 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:47.118473 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.118473 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:47.125376 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] OVN maintenance task set_network_type_and_physnet finished in 0.212 seconds
Apr 21 13:29:47.127162 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_lrouter_ports_ext_ids_name_prefix finished in 0.084 seconds
Apr 21 13:29:47.132023 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f50ab82-a48b-4b7c-954b-0ad7049f328b None None] OVN maintenance task check_provider_distributed_ports finished in 0.494 seconds
Apr 21 13:29:47.135610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Mark Neutron-owned static routes in OVN external_ids. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:47.138610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' {{(pid=106267) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 21 13:29:47.139071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_security_group_with_address_group {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:47.142710 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:47.156619 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:29:47.376462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:47.377879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:29:47.377879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.377879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:29:47.377879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.377879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.377879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:29:47.377879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.377879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:29:47.378722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:47.378849 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:47.388799 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [None req-6f50ab82-a48b-4b7c-954b-0ad7049f328b None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:47.399757 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a630e31f-727d-412e-b2c8-45dace5792f5 None None] OVN maintenance task update_router_distributed_flag finished in 0.020 seconds
Apr 21 13:29:47.405305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Repair Address_Sets and ACLs for address group rules. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:47.407337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.409464 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task update_router_static_routes finished in 0.019 seconds
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.433 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:47.427368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.427733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.428141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:47.428141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.428141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:47.431467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [None req-a630e31f-727d-412e-b2c8-45dace5792f5 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:47.432034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:47.433951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Traceback (most recent call last):
Apr 21 13:29:47.433951 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.433951 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:47.433951 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.433951 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.433951 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:47.433951 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.433951 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:47.436981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:47.441921 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.449287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:47.450656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.451880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.451880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:47.451880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.451880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task check_fdb_aging_settings failed after 0.965 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:47.454769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.455317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:47.455667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.455667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:47.456481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:47.457171 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_security_group_with_address_group finished in 0.044 seconds
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:47.460794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.461856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.462548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:47.462548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.462548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:47.462548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.464805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:47.465357 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_redirect_type_router_gateway_ports finished in 0.791 seconds
Apr 21 13:29:47.467028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:47.467028 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.467028 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:47.467028 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.467028 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.467028 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:47.467028 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.467028 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.424 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.469163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:47.469568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.469976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:47.469976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106267) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:47.472081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.472931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:47.473813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.473813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:47.484504 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=25c0f690-97c0-477a-80e9-c7b53284c042, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-b57c058f-054e-4a2a-980f-6bfda2111e90', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-40826ab9-f311-4f0d-921e-dee16be9cb42', 'neutron:subnet_ids': 'c97deb27-a1d6-4e03-b489-95abde8fd791'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={'address_mode': 'slaac', 'mtu': '1372', 'send_periodic': 'true'}, mac='fa:16:3e:3f:51:93', name='lrp-d461897d-bdb7-49f1-b874-9a6c4530f7cc', networks=['fd64:7955:a2a8::1/64'], options={}, peer=[], status={}) old= {{(pid=106267) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 21 13:29:47.490725 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Apr 21 13:29:47.580612 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-b57c058f-054e-4a2a-980f-6bfda2111e90 synchronized; highest priority chassis 27f467c0-a74a-4bc5-9cac-d4ea2716e85a
Apr 21 13:29:47.582726 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': 'b57c058f-054e-4a2a-980f-6bfda2111e90', 'neutron:router_id': '40826ab9-f311-4f0d-921e-dee16be9cb42'}),), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:47.583022 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, chassis=27f467c0-a74a-4bc5-9cac-d4ea2716e85a, priority=1, columns={}) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:47.583234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.583851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:47.583851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.583851 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:47.583851 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.583851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.583851 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:47.583851 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.583851 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2411, in link_network_ha_chassis_group
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     next(self.gen)
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     next(self.gen)
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 21 13:29:47.587206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 21 13:29:47.588184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:47.588184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     raise result.ex
Apr 21 13:29:47.588184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:47.588184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 21 13:29:47.588184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 21 13:29:47.588184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:47.588184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 21 13:29:47.588184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:47.588184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event 
Apr 21 13:29:47.644598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 21 13:29:47.644856 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-412379ac-0333-47e4-9d03-fc1dfbaa1237 None None] OVN-Southbound DB sync process completed @ 2026-04-21 13:29:47.644659 {{(pid=106267) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1482}}
Apr 21 13:29:49.429439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=8a347862-bc06-45a7-b62e-90f9015f267a, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-d0607710-681d-444d-be26-dbeca5a07b46', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-40826ab9-f311-4f0d-921e-dee16be9cb42', 'neutron:subnet_ids': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'}, gateway_chassis=[], ha_chassis_group=1085f663-fb79-4c9a-8984-1885b04896da, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:fa:68:66', name='lrp-f2518dca-5ef4-4d8e-888f-17ddc84df4d1', networks=['172.24.5.75/24'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={'hosting-chassis': '27f467c0-a74a-4bc5-9cac-d4ea2716e85a'}) old= {{(pid=106267) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 21 13:29:49.430526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=d775e6de-705e-4ffb-a9a4-78024beba5fe, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-d0607710-681d-444d-be26-dbeca5a07b46', 'neutron:revision_number': '8', 'neutron:router_name': 'neutron-40826ab9-f311-4f0d-921e-dee16be9cb42', 'neutron:subnet_ids': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3 ca2ae363-6d7d-4b39-b330-44fb15b7cd49'}, gateway_chassis=[], ha_chassis_group=1085f663-fb79-4c9a-8984-1885b04896da, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:fa:68:66', name='lrp-f2518dca-5ef4-4d8e-888f-17ddc84df4d1', networks=['172.24.5.75/24', '2001:db8::1/64'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=106267) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 21 13:29:49.471664 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-b57c058f-054e-4a2a-980f-6bfda2111e90 synchronized; highest priority chassis 27f467c0-a74a-4bc5-9cac-d4ea2716e85a
Apr 21 13:29:49.472439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': 'b57c058f-054e-4a2a-980f-6bfda2111e90', 'neutron:router_id': '40826ab9-f311-4f0d-921e-dee16be9cb42'}),), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:49.472959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, chassis=27f467c0-a74a-4bc5-9cac-d4ea2716e85a, priority=1, columns={}) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:49.473246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:49.473976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:49.473976 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:49.473976 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:49.473976 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:49.473976 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:49.473976 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:49.473976 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:49.473976 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 81, in run
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1664, in update_router_ha_chassis_group
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2411, in link_network_ha_chassis_group
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     next(self.gen)
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     next(self.gen)
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:49.475044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     raise result.ex
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:49.475848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event 
Apr 21 13:29:49.508658 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-b57c058f-054e-4a2a-980f-6bfda2111e90 synchronized; highest priority chassis 27f467c0-a74a-4bc5-9cac-d4ea2716e85a
Apr 21 13:29:49.509698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': 'b57c058f-054e-4a2a-980f-6bfda2111e90', 'neutron:router_id': '40826ab9-f311-4f0d-921e-dee16be9cb42'}),), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:49.510119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, chassis=27f467c0-a74a-4bc5-9cac-d4ea2716e85a, priority=1, columns={}) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:49.510422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:49.511616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:49.511616 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:49.511616 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:49.511616 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:49.511616 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:49.511616 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:49.511616 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:49.511616 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 89, in run
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1664, in update_router_ha_chassis_group
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2411, in link_network_ha_chassis_group
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     next(self.gen)
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     next(self.gen)
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:49.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     raise result.ex
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:49.513932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event 
Apr 21 13:29:51.421269 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:29:51.422034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a630e31f-727d-412e-b2c8-45dace5792f5 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:51.422149 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a630e31f-727d-412e-b2c8-45dace5792f5 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:51.423455 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a630e31f-727d-412e-b2c8-45dace5792f5 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:29:51.457335 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:51.457860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.458777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a630e31f-727d-412e-b2c8-45dace5792f5 None None] Traceback (most recent call last):
Apr 21 13:29:51.458777 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.458777 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:51.458777 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.458777 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.458777 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:51.458777 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.458777 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a630e31f-727d-412e-b2c8-45dace5792f5 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.459849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.460429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a630e31f-727d-412e-b2c8-45dace5792f5 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:51.461242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:51.461936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.462762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:51.462762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.462762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.462762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:51.462762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.462762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:51.616886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:29:51.618255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:51.618255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:51.619019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:51.619330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.619973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:51.619973 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.619973 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:51.619973 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.619973 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.619973 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:51.619973 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.619973 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.621585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.621897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:51.622266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.622266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.623859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:51.624356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.624356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:51.770083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:29:51.771000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:51.771192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:51.771365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:29:51.772184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:51.772611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.773593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:51.773593 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.773593 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:51.773593 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.773593 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.773593 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:51.773593 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.773593 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:29:51.774957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.775953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:29:51.775953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.775953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:29:51.775953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.775953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.775953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:29:51.775953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.775953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.776451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:51.777200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.778103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:51.778717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.779801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:51.779801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:51.779801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.779801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:51.779801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.779801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:51.779801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.779801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.779801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:51.779801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.779801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:51.911152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:29:51.911954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:51.912130 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:51.912130 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:29:51.912748 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:51.913038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.913762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:51.913762 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.913762 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:51.913762 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.913762 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.913762 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:51.913762 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.913762 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:29:51.915032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.915797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:29:51.915797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.915797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:29:51.915797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.915797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.915797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:29:51.915797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.915797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:51.917594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.918151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:51.918869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:51.919598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.920687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:51.920687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.920687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:51.920687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.920687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.920687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:51.920687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.920687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:51.989833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:29:51.990490 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:51.990594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:51.990784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:29:51.991389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:51.991773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:51.991986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.992753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:51.992753 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.992753 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:51.992753 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.992753 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.992753 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:51.992753 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.992753 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.993817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:51.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:51.995281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:51.996305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:51.997185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:51.997185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:51.997185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:56.422770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:29:56.424953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:56.424953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:56.426085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:29:56.454859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:56.455349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.456378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Traceback (most recent call last):
Apr 21 13:29:56.456378 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.456378 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:56.456378 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.456378 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.456378 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:56.456378 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.456378 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.457429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.458027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:56.458632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:56.459421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.460372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:56.460372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.460372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.460372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:56.460372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.460372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:56.618076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:29:56.619694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:56.619847 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:56.620849 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:56.621171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.622031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:56.622031 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.622031 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:56.622031 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.622031 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.622031 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:56.622031 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.622031 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.623960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.624557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:56.625112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.625112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.626024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.626984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:56.627859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.627859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:56.771066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:29:56.773298 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:56.773397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:56.773512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:29:56.774742 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:56.775068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.775834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:56.775834 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.775834 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:56.775834 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.775834 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.775834 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:56.775834 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.775834 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:29:56.776741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.777270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:29:56.777270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.777270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:29:56.777270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.777270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.777270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:29:56.777270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.777270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.777664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:56.778021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.778443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.778443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:56.778443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.778443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.778719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:56.779243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.779822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.779822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:56.779822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.779822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:56.912250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:29:56.913374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:56.913455 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:56.913598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:29:56.914111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:56.914541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.915272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:56.915272 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.915272 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:56.915272 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.915272 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.915272 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:56.915272 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.915272 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:29:56.916060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.917121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:56.918392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:56.919506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:56.920113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.921067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:29:56.991689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:29:56.991838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:29:56.991931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:29:56.992120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:29:56.992897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:56.993370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:29:56.993566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.994410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:29:56.994410 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.994410 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:29:56.994410 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.994410 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.994410 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:29:56.994410 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.994410 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.995814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:29:56.996482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.997109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:29:56.997737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:29:56.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:29:56.999400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:29:56.999400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:01.423985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:01.424914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:01.425002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:01.425867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:01.453656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:01.454133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.455337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Traceback (most recent call last):
Apr 21 13:30:01.455337 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.455337 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:01.455337 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.455337 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.455337 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:01.455337 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.455337 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.456360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.456961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:01.457571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:01.458271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.459385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:01.459385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.459385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.459385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:01.459385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.459385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:01.619753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:01.620696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:01.620857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:01.621513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:01.621752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.622399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:01.622399 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.622399 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:01.622399 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.622399 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.622399 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:01.622399 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.622399 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.623382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.623972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:01.624645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.624645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.625085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.625939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:01.626864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.626864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:01.773502 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:01.774462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:01.774577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:01.774722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:01.775326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:01.775637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.776472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:01.776472 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.776472 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:01.776472 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.776472 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.776472 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:01.776472 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.776472 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:01.777779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.778553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:01.778553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.778553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:01.778553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.778553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.778553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:01.778553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.778553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.778917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:01.779659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.780403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:01.781124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:01.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:01.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:01.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:01.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:01.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:01.913510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:01.914416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:01.914553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:01.914708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:01.915593 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:01.916102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.917106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:01.917106 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.917106 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:01.917106 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.917106 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.917106 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:01.917106 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.917106 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:01.918380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.919066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:01.919066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.919066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:01.919066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.919066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.919066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:01.919066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.919066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:01.920250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.920824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:01.922099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:01.922888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.923893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:01.923893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.923893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:01.923893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.923893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.923893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:01.923893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.923893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:01.992022 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:01.993320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:01.993442 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:01.993642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:01.994175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:01.994578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:01.994751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.995476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:01.995476 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.995476 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:01.995476 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.995476 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.995476 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:01.995476 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.995476 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.996493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:01.997124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:01.997658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:01.998411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:01.999257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:01.999257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:01.999257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:06.425176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:06.426709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:06.426878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:06.427768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:06.460322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:06.460322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:06.462273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:06.463121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.463734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:06.464333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:06.465269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:06.465269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:06.465269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.465269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:06.465269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.465269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.465269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:06.465269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.465269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:06.620962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:06.621848 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:06.621941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:06.625444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:06.626401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:06.627116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.627762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.628673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:06.774742 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:06.776287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:06.776287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:06.776287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:06.776754 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:06.777146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.778059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:06.778059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.778059 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:06.778059 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.778059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.778059 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:06.778059 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.778059 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:06.779386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:06.780145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:06.780145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.780145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:06.780145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.780145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.780145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:06.780145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.780145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.780692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:06.781310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.781855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.781855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:06.781855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.781855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.783293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:06.783820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.784403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.784403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:06.784403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.784403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:06.914706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:06.915779 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:06.915901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:06.915968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:06.916605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:06.917004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.917891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:06.917891 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.917891 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:06.917891 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.917891 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.917891 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:06.917891 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.917891 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:06.918813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:06.919536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:06.919536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.919536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:06.919536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.919536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.919536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:06.919536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.919536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:06.919910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.920482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:06.921080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:06.921822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:06.922791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:06.922791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.922791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:06.922791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.922791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.922791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:06.922791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.922791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:06.993737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:06.994633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:06.994767 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:06.994982 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:06.995596 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:06.995984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:06.996236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.996995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:06.996995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.996995 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:06.996995 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.996995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.996995 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:06.996995 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.996995 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:06.998397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:06.999022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.999598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:06.999927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:07.000734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:07.001628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:07.001628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:11.428623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:11.429587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:11.429701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:11.430560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:11.503473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:11.503683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.504370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:30:11.504370 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.504370 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:11.504370 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.504370 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.504370 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:11.504370 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.504370 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.075 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:11.505476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.506165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:11.506549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:11.506865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.507305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:11.507305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.507305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.507305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:11.507305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.507305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:11.622188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:11.623274 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:11.623368 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:11.624181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:11.624748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.626191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:11.626191 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.626191 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:11.626191 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.626191 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.626191 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:11.626191 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.626191 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:11.628170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.628758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:11.629361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.629361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.631387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.632303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:11.633151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.633151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:11.775817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:11.777066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:11.777229 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:11.777394 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:11.777961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:11.778320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.779336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:11.779336 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.779336 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:11.779336 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.779336 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.779336 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:11.779336 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.779336 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:11.780945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:11.782099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:11.782099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.782099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:11.782099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.782099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.782099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:11.782099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.782099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.782451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:11.782831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.783244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.783244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:11.783244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.783244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.784063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:11.784604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.785199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.785199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:11.785199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.785199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:11.916010 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:11.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:11.917044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:11.917236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:11.917845 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:11.918122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.918751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:11.918751 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.918751 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:11.918751 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.918751 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.918751 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:11.918751 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.918751 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:11.920053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:11.920789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:11.920789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.920789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:11.920789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.920789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.920789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:11.920789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.920789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:11.921438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.922011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.922642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:11.922642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.922642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:11.923228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:11.924016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.924833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.924833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:11.924833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.924833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:11.994619 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:11.995564 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:11.995711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:11.995796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:11.996443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:11.996834 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:11.997026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.999375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:11.999375 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:11.999375 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:11.999375 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:11.999375 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:11.999375 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:11.999375 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:11.999375 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:12.000153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:12.000849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:12.001241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:12.001733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:12.002297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:12.002297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:12.002297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:16.433268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:16.433268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:16.433268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:16.433268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:16.465494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:16.465841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.466849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:30:16.466849 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.466849 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:16.466849 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.466849 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.466849 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:16.466849 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.466849 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:16.467609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.468234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:16.468875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:16.469689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.470912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:16.470912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.470912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.470912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:16.470912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.470912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:16.623324 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:16.625716 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:16.625825 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:16.626933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:16.627378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.628239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:16.628239 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.628239 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:16.628239 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.628239 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.628239 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:16.628239 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.628239 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:16.629702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.630371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:16.631078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.631078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.631371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.632192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:16.633049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.633049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:16.777323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:16.778611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:16.778797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:16.778959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:16.779741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:16.780076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.780891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:16.780891 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.780891 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:16.780891 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.780891 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.780891 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:16.780891 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.780891 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:16.782232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:16.783035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:16.783035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.783035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:16.783035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.783035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.783035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:16.783035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.783035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.783525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:16.783843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.784284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.784284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:16.784284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.784284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.786392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:16.786974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.787602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.787602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:16.787602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.787602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:16.917309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:16.918204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:16.918341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:16.918501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:16.919386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:16.919799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.920604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:16.920604 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.920604 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:16.920604 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.920604 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.920604 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:16.920604 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.920604 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:16.921635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:16.922305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:16.922305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.922305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:16.922305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.922305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.922305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:16.922305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.922305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:16.922903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.923428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.923990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:16.923990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.923990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:16.924551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:16.925075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:16.925631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:16.925631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:16.925631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:16.925631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:16.995926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:16.997409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:16.997409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:16.997409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:16.998008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:16.998442 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:16.998682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:17.002251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:17.003071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:17.003492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:17.003492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:17.003492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:17.003492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:17.003492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:17.003833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:17.004330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:17.004822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:17.004822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:21.431856 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:21.433049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:21.433173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:21.434284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:21.467362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:21.467624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.468275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Traceback (most recent call last):
Apr 21 13:30:21.468275 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.468275 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:21.468275 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.468275 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.468275 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:21.468275 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.468275 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:21.469050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.469751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:21.470350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:21.470959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.471741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:21.471741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.471741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.471741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:21.471741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.471741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:21.625795 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:21.626827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:21.627014 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:21.628106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:21.628450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.629426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:21.629426 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.629426 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:21.629426 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.629426 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.629426 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:21.629426 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.629426 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:21.631044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.631436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:21.631793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.631793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.631988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.632487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:21.633005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.633005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:21.778597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:21.779426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:21.779582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:21.779749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:21.780549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:21.780870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.783080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:21.783080 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.783080 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:21.783080 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.783080 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.783080 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:21.783080 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.783080 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:21.783990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:21.784743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:21.784743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.784743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:21.784743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.784743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.784743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:21.784743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.784743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.785098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:21.785654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.786281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:21.786665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:21.787145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:21.787145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:21.787145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:21.787145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:21.787145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.787145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:21.787145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.787145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.787145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:21.787145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.787145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:21.918534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:21.920028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:21.920235 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:21.920473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:21.921325 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:21.922031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.922935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:21.922935 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.922935 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:21.922935 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.922935 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.922935 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:21.922935 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.922935 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:21.924230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:21.925166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:21.925166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.925166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:21.925166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.925166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.925166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:21.925166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.925166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:21.925887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.927499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:21.927499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.927499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:21.928302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:21.929419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:21.930607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:21.930607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:21.930607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:21.930607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:21.997279 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:21.998403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:21.998552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:21.998787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:21.999510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:21.999921 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:22.000162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:22.001026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:22.001026 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:22.001026 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:22.001026 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:22.001026 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:22.001026 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:22.001026 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:22.001026 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:22.002571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:22.002981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:22.003388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:22.004241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:22.004758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:22.005328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:22.005328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:26.433850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:26.436747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:26.437067 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:26.438795 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:26.467137 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:26.467572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.468378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:30:26.468378 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.468378 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:26.468378 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.468378 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.468378 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:26.468378 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.468378 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:26.469168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.469554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:26.469920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.469920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:26.469987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.470454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.470953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:26.470953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.470953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:26.627034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:26.628542 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:26.628704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:26.629815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:26.630147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.631574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:26.631574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.631574 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:26.631574 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.631574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.631574 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:26.631574 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.631574 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:26.632825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.633400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:26.634159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.634159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.635310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.635771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:26.636261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.636261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:26.779885 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:26.780512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:26.780601 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:26.780663 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:26.781333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:26.781707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.782543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:26.782543 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.782543 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:26.782543 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.782543 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.782543 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:26.782543 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.782543 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:26.783927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:26.784635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:26.784635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.784635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:26.784635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.784635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.784635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:26.784635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.784635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.785239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:26.785785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.786529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.786529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:26.786529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.786529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.786757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:26.787502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.789263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.789263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:26.789263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.789263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:26.920501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:26.921526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:26.921681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:26.921906 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:26.922497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:26.922796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.923440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:26.923440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.923440 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:26.923440 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.923440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.923440 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:26.923440 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.923440 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:26.924771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:26.925708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:26.925708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.925708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:26.925708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.925708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.925708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:26.925708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.925708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:26.926796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.927552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.928236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:26.928236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.928236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:26.929341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:26.929924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:26.930591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:26.930591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:26.930591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:26.930591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:26.998982 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:26.999776 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:26.999914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:27.000175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:27.000762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:27.001057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:27.001169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:27.001883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:27.001883 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:27.001883 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:27.001883 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:27.001883 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:27.001883 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:27.001883 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:27.001883 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:27.003775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:27.004560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:27.005230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:27.005445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:27.006297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:27.006822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:27.006822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:31.437044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:31.439132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:31.439259 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:31.440084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:31.469638 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:31.469991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.470931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:30:31.470931 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.470931 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:31.470931 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.470931 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.470931 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:31.470931 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.470931 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:31.471879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.472563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:31.473264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.473264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:31.474304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.476478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:31.476478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.476478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:31.629068 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:31.630045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:31.630224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:31.631297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:31.631651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.632630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:31.632630 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.632630 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:31.632630 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.632630 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.632630 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:31.632630 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.632630 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:31.634149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.634746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:31.635315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.635315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.635880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.636745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:31.637597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.637597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:31.780963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:31.782876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:31.782876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:31.782876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:31.783509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:31.783862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.785492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:31.785492 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.785492 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:31.785492 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.785492 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.785492 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:31.785492 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.785492 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:31.786771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:31.787636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:31.787636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.787636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:31.787636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.787636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.787636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:31.787636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.787636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.789530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:31.790128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.790733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.790733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:31.790733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.790733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.792732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:31.793573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.794477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.794477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:31.794477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.794477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:31.921665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:31.923131 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:31.923444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:31.923642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:31.924468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:31.924973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.925840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:31.925840 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.925840 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:31.925840 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.925840 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.925840 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:31.925840 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.925840 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:31.927508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:31.928319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:31.928319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.928319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:31.928319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.928319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.928319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:31.928319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.928319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:31.929161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.929722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.930372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:31.930372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.930372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:31.931165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:31.931971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:31.932848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:31.932848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:31.932848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:31.932848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:31.999907 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:32.001107 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:32.001250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:32.001566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:32.002294 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:32.002732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:32.002978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:32.003925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:32.003925 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:32.003925 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:32.003925 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:32.003925 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:32.003925 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:32.003925 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:32.003925 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:32.005522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:32.006193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:32.006832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:32.007713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:32.008684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:32.009455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:32.009455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:36.439407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:36.440623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:36.440899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:36.441992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:36.444351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=577,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:30:36.478566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:36.479047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.480041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Traceback (most recent call last):
Apr 21 13:30:36.480041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.480041 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:36.480041 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.480041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.480041 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:36.480041 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.480041 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:36.481076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:36.482183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.482183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:36.482314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.483050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.483811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:36.483811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.483811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:36.630423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:36.631422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:36.631567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:36.632456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:36.632835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.633685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:36.633685 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.633685 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:36.633685 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.633685 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.633685 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:36.633685 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.633685 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:36.635464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.635848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:36.636191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.636191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.637958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.638438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:36.638902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.638902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:36.782547 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:36.783588 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:36.783877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:36.784243 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:36.785031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:36.785460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.786352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:36.786352 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.786352 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:36.786352 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.786352 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.786352 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:36.786352 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.786352 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:36.788153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:36.789313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:36.789313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.789313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:36.789313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.789313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.789313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:36.789313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.789313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.790757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:36.791575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.792509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.792509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:36.792509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.792509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.793336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:36.794472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.795476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.795476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:36.795476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.795476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:36.922892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:36.924312 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:36.924439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:36.924715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:36.925433 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:36.925866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.926765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:36.926765 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.926765 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:36.926765 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.926765 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.926765 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:36.926765 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.926765 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:36.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.928648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:36.929482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:36.930336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:36.931177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:36.932303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:37.001438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:37.002540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:37.002707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:37.002953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:37.003599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:37.004023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:37.004219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:37.005086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:37.005086 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:37.005086 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:37.005086 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:37.005086 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:37.005086 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:37.005086 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:37.005086 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:37.007000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:37.007451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:37.007835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:37.009642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:37.010204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:37.010801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:37.010801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:41.848573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:41.849305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:41.850326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:41.850430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:41.852364 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:41.852780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:41.854577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:41.855128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:41.855256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:41.856761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:41.857034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:41.857255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:41.867459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:41.868134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.868703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:41.869229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:41.870002 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:41.872642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:41.874378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:41.875446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.876156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.876553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:41.876972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.877472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.877472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:41.877472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.877472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:41.893149 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:41.893576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.894385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Traceback (most recent call last):
Apr 21 13:30:41.894385 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.894385 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:41.894385 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.894385 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.894385 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:41.894385 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.894385 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:41.895059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.895726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:41.896543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:41.897298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.898193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:41.898193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.898193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.898193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:41.898193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.898193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:41.924241 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:41.925317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:41.925466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:41.925594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:41.926229 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:41.926513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.927085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:41.927085 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.927085 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:41.927085 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.927085 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.927085 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:41.927085 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.927085 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:41.928387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:41.929144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:41.929144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.929144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:41.929144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.929144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.929144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:41.929144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.929144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:41.929800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.930436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.931071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:41.931071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.931071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:41.931489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:41.932337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:41.933193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:41.933193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:41.933193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:41.933193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:42.002499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:42.003633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:42.003847 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:42.004098 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:42.004852 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:42.005310 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:42.005572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:42.006424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:42.006424 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:42.006424 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:42.006424 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:42.006424 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:42.006424 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:42.006424 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:42.006424 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:42.007982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:42.008746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:42.009383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:42.009679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:42.010674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:42.011622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:42.011622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:46.850525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:46.851597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:46.851597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:46.853599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:46.854342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:46.855596 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:46.856247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:46.858163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:46.858551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.859635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Traceback (most recent call last):
Apr 21 13:30:46.859635 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.859635 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:46.859635 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.859635 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.859635 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:46.859635 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.859635 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:46.860615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:46.865504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.866600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:46.866975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.866975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:46.868418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.868861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.869345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:46.869345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.869345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:46.869345 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:46.870253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:46.870450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:46.871072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:46.871432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.872523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:46.872523 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.872523 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:46.872523 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.872523 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.872523 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:46.872523 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.872523 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:46.874937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:46.875424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:46.875424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.875424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:46.875424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.875424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.875424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:46.875424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.875424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.879434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:46.884970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.885639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:46.886413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:46.887329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:46.887329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:46.887329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:46.887329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:46.887329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.887329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:46.887329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.887329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.887329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:46.887329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.887329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:46.903366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:46.903649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.904422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Traceback (most recent call last):
Apr 21 13:30:46.904422 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.904422 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:46.904422 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.904422 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.904422 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:46.904422 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.904422 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:46.905507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.906350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:46.906970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:46.907881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.908942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:46.908942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.908942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.908942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:46.908942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.908942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:46.925287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:46.925618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:46.926060 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:46.926282 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:46.927006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:46.927573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.928541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:46.928541 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.928541 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:46.928541 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.928541 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.928541 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:46.928541 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.928541 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:46.930022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:46.930496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:46.930496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.930496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:46.930496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.930496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.930496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:46.930496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.930496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:46.931588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.932686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:46.933249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:46.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:46.934895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:46.934895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:46.934895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:46.934895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:46.934895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:46.934895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:46.934895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:46.934895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:47.003568 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:47.003865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:47.003985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:47.004239 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:47.005351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:47.005672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:47.005865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:47.006584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:47.006584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:47.006584 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:47.006584 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:47.006584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:47.006584 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:47.006584 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:47.006584 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:47.007922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:47.008609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:47.009267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:47.009437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:47.010086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:47.010633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:47.010633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:51.851460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:51.852559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:51.852559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:51.853281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:51.856704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:51.857114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:51.857114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:51.859109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:51.859507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.861403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:51.861403 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.861403 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:51.861403 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.861403 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.861403 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:51.861403 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.861403 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:51.866795 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:51.870065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.870541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:51.871001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.871001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.871377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.872334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:51.873323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.873323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:51.873323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:51.873323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:51.873323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:51.873646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:51.873799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.874711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:51.874711 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.874711 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:51.874711 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.874711 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.874711 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:51.874711 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.874711 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:51.876556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:51.877518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:51.877518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.877518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:51.877518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.877518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.877518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:51.877518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.877518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.880463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:51.880872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.881369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.881369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:51.881369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.881369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.883600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:51.884137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.884725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.884725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:51.884725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.884725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:51.902869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:51.903295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.904323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Traceback (most recent call last):
Apr 21 13:30:51.904323 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.904323 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:51.904323 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.904323 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.904323 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:51.904323 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.904323 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:51.905167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.905921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:51.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:51.907290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.908112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:51.908112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.908112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.908112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:51.908112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.908112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:51.926056 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:51.926596 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:51.926731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:51.926953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:51.927793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:51.928152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.928943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:51.928943 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.928943 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:51.928943 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.928943 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.928943 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:51.928943 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.928943 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:51.931010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:51.931550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:51.931550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.931550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:51.931550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.931550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.931550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:51.931550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.931550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:51.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.933393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.934063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:51.934063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.934063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:51.935556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:51.936781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:51.937731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:51.937731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:51.937731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:51.937731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:52.004278 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:52.004818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:52.004818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:52.005034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:52.005496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:52.005796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:52.005934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:52.006659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:52.006659 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:52.006659 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:52.006659 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:52.006659 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:52.006659 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:52.006659 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:52.006659 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:52.007642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:52.008037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:52.008420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:52.008706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:52.009539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:52.010042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:52.010042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:56.852340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:56.853278 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:56.853437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:56.854682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:56.857154 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:56.857883 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:56.858028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:56.859822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:56.863586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.864629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:56.864629 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.864629 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:56.864629 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.864629 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.864629 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:56.864629 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.864629 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:56.867667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.868565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:56.869187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.869187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.869637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.871092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:56.873522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.873522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:56.873522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:56.873522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:56.873522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:56.873522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:30:56.874033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:56.874238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.874824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:56.874824 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.874824 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:56.874824 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.874824 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.874824 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:56.874824 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.874824 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:56.876619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:56.877074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:56.877074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.877074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:56.877074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.877074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.877074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:56.877074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.877074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.878014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:56.878278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.878523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.878523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:56.878523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.878523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.882056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:56.882400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.882798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.882798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:56.882798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.882798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:56.901483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:56.901705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.902324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Traceback (most recent call last):
Apr 21 13:30:56.902324 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.902324 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:56.902324 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.902324 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.902324 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:56.902324 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.902324 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:56.902964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.903583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:56.904142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:56.904856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.905720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:56.905720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.905720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.905720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:56.905720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.905720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:56.926550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:56.926903 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:56.927045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:56.927217 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:56.927644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:56.928008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.928965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:56.928965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.928965 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:56.928965 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.928965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.928965 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:56.928965 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.928965 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:30:56.930186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:56.930636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:30:56.930636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.930636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:30:56.930636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.930636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.930636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:30:56.930636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.930636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:56.931717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.932079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.932486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:56.932486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.932486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:56.933685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:56.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:56.934714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:56.934714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:56.934714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:56.934714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:30:57.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:30:57.005346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:30:57.005484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:30:57.005700 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:30:57.006179 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:57.006585 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:30:57.006776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:57.007501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:30:57.007501 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:57.007501 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:30:57.007501 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:30:57.007501 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:57.007501 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:30:57.007501 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:57.007501 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:57.009054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:30:57.009650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:57.010236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:30:57.010785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:30:57.011548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:30:57.012639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:30:57.012639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:01.853178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:01.854178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:01.854178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:01.855352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:01.858057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:01.858759 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:01.858919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:01.860674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:01.864234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.866235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:01.866235 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.866235 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:01.866235 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.866235 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.866235 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:01.866235 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.866235 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:01.869557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.870322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:01.870904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.870904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.872161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.873079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:01.873942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.873942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:01.875511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:01.876686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:01.876811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:01.876928 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:01.878467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:01.878883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.880630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:01.880630 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.880630 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:01.880630 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.880630 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.880630 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:01.880630 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.880630 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:01.882489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:01.883000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:01.883000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.883000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:01.883000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.883000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.883000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:01.883000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.883000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.884286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:01.885020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.885850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.885850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:01.885850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.885850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.886580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:01.887851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.889200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.889200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:01.889200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.889200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:01.899689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:01.899969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.900643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Traceback (most recent call last):
Apr 21 13:31:01.900643 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.900643 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:01.900643 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.900643 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.900643 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:01.900643 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.900643 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:01.901626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.902042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:01.902418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.902418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:01.902762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.903251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.904038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:01.904038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.904038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:01.943919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:01.943919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:01.943919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:01.943919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:01.945299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:01.945511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.946294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:01.946294 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.946294 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:01.946294 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.946294 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.946294 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:01.946294 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.946294 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:01.947596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:01.948020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:01.948020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.948020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:01.948020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.948020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.948020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:01.948020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.948020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.018 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:01.948602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.949194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.949804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:01.949804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.949804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:01.950193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:01.950946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:01.951744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:01.951744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:01.951744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:01.951744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:02.005504 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:02.005933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:02.006099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:02.006453 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:02.007232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:02.007857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:02.008053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:02.008783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:02.008783 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:02.008783 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:02.008783 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:02.008783 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:02.008783 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:02.008783 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:02.008783 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:02.010328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:02.010719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:02.011109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:02.012458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:02.013356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:02.013917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:02.013917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:06.853995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:06.855870 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:06.856008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:06.857443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:06.858742 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:06.859305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:06.859474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:06.865146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:06.865385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.869276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:06.869276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.869276 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:06.869276 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.869276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.869276 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:06.869276 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.869276 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:06.872045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.872736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:06.873364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.873364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.876302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.877240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:06.878095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.878095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:06.878095 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:06.879943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:06.880958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:06.881113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:06.882476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:06.882881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.884658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:06.884658 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.884658 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:06.884658 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.884658 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.884658 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:06.884658 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.884658 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:06.886858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:06.887612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:06.887612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.887612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:06.887612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.887612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.887612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:06.887612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.887612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.888234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:06.888822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.889493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.889493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:06.889493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.889493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.890715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:06.891468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.892243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.892243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:06.892243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.892243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:06.909083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:06.909300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.909919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:31:06.909919 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.909919 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:06.909919 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.909919 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.909919 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:06.909919 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.909919 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:06.911102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.911546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:06.912096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.912096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:06.912199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.912978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.913945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:06.913945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.913945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:06.930006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:06.930358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:06.930492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:06.930691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:06.931427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:06.931904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.932850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:06.932850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.932850 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:06.932850 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.932850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.932850 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:06.932850 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.932850 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:06.934169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:06.935101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:06.935101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.935101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:06.935101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.935101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.935101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:06.935101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.935101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:06.935631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.936246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.936915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:06.936915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.936915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:06.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:06.938494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:06.939282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:06.939282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:06.939282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:06.939282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:07.006075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:07.006683 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:07.006827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:07.007057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:07.007768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:07.008231 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:07.008497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:07.009343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:07.009343 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:07.009343 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:07.009343 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:07.009343 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:07.009343 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:07.009343 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:07.009343 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:07.010849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:07.011525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:07.012161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:07.012496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:07.013262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:07.014077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:07.014077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:11.856116 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:11.856930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:11.856930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:11.858368 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:11.859635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:11.860887 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:11.861028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:11.862079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:11.862502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.867484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:11.867484 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.867484 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:11.867484 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.867484 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.867484 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:11.867484 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.867484 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:11.870517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.870924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:11.871306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.871306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.872659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.873171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:11.873812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.873812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:11.881441 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:11.881823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:11.881955 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:11.881955 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:11.882617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:11.883000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.883774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:11.883774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.883774 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:11.883774 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.883774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.883774 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:11.883774 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.883774 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:11.885643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:11.886131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:11.886131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.886131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:11.886131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.886131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.886131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:11.886131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.886131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.886617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:11.886949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.887333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.887333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:11.887333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.887333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.889244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:11.889828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.890383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.890383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:11.890383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.890383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:11.924521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:11.925029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.926034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:31:11.926034 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.926034 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:11.926034 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.926034 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.926034 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:11.926034 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.926034 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task check_fdb_aging_settings failed after 0.069 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:11.927188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.927761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:11.928323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.928323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:11.928425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.929256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.930064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:11.930064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.930064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:11.930346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:11.930791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:11.930901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:11.931058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:11.931481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:11.931720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.932276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:11.932276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.932276 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:11.932276 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.932276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.932276 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:11.932276 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.932276 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:11.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:11.934077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:11.934077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.934077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:11.934077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.934077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.934077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:11.934077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.934077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:11.934400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.934735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.935106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:11.935106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.935106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:11.935542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:11.936099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:11.936904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:11.936904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:11.936904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:11.936904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:12.006765 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:12.007374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:12.007481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:12.007726 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:12.008451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:12.008904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:12.009096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:12.009976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:12.009976 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:12.009976 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:12.009976 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:12.009976 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:12.009976 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:12.009976 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:12.009976 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:12.010952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:12.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:12.012234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:12.012935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:12.013852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:12.013852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:12.013852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:16.857039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:16.858014 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:16.858014 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:16.859376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:16.861035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:16.862678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:16.863063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:16.864166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:16.864587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.865765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:16.865765 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.865765 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:16.865765 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.865765 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.865765 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:16.865765 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.865765 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:16.868392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.869067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:16.869699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.869699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.870099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.870919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:16.872185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.872185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:16.882051 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:16.882374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:16.882437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:16.882556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:16.883054 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:16.883412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.884161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:16.884161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.884161 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:16.884161 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.884161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.884161 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:16.884161 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.884161 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:16.885188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:16.885611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:16.885611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.885611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:16.885611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.885611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.885611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:16.885611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.885611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.886102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:16.886634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.887234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.887234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:16.887234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.887234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.888240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:16.888945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.889720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.889720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:16.889720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.889720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:16.930992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:16.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:16.931625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:16.931797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:16.932513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:16.932864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.936138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:16.936138 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.936138 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:16.936138 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.936138 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.936138 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:16.936138 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.936138 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:16.939316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:16.940261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:16.940261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.940261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:16.940261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.940261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.940261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:16.940261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.940261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:16.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.941152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.941713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:16.941713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.941713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.942145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:16.942838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.943357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.943357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:16.943357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.943357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:16.946923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:16.947345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.948879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Traceback (most recent call last):
Apr 21 13:31:16.948879 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.948879 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:16.948879 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.948879 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.948879 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:16.948879 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.948879 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] OVN maintenance task check_fdb_aging_settings failed after 0.091 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:16.950125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.950860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:16.951463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:16.952273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:16.953107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:16.953107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:16.953107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:16.953107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:16.953107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:16.953107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:17.007506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:17.008177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:17.008433 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:17.008888 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:17.009711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:17.010195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:17.010486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:17.011374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:17.011374 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:17.011374 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:17.011374 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:17.011374 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:17.011374 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:17.011374 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:17.011374 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:17.013206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:17.013796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:17.014342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:17.015354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:17.016203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:17.017245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:17.017245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:21.857639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:21.858256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:21.858256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:21.858894 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:21.864981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:21.866477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:21.866770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:21.867933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:21.869620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.870763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:21.870763 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.870763 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:21.870763 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.870763 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.870763 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:21.870763 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.870763 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:21.873435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.874042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:21.874990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.874990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.875489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.877370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:21.878597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.878597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:21.882502 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:21.883793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:21.883998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:21.885047 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:21.886646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:21.888288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.889149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:21.889149 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.889149 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:21.889149 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.889149 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.889149 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:21.889149 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.889149 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:21.891656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:21.892311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:21.892311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.892311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:21.892311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.892311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.892311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:21.892311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.892311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.894319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:21.895170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.895905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.895905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:21.895905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.895905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.896486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:21.898376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.899788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.899788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:21.899788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.899788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:21.907251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:21.907633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.908462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Traceback (most recent call last):
Apr 21 13:31:21.908462 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.908462 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:21.908462 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.908462 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.908462 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:21.908462 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.908462 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:21.909555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.910417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:21.911545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:21.912565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.913851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:21.913851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.913851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.913851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:21.913851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.913851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:21.931728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:21.932016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:21.932150 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:21.932336 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:21.932832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:21.933185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.933923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:21.933923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.933923 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:21.933923 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.933923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.933923 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:21.933923 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.933923 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:21.935315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:21.936080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:21.936080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.936080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:21.936080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.936080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.936080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:21.936080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.936080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:21.936773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.938126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:21.938126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.938126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:21.938624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:21.939561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:21.940174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:21.940174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:21.940174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:21.940174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:22.031570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:22.032377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:22.032425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:22.032568 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:22.032891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:22.033077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:22.033162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:22.033572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:22.033572 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:22.033572 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:22.033572 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:22.033572 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:22.033572 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:22.033572 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:22.033572 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:22.034289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:22.034542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:22.034824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:22.035099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:22.035475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:22.035852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:22.035852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:26.858406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:26.859075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:26.859075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:26.859616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:26.868597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:26.869829 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:26.869983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:26.871373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:26.871807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.877671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:26.877671 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.877671 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:26.877671 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.877671 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.877671 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:26.877671 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.877671 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:26.879678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.880246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:26.880784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.880784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.884179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.884952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:26.885768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.885768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:26.885768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:26.886313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:26.886416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:26.886508 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:26.887687 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:26.888014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.892616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:26.892616 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.892616 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:26.892616 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.892616 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.892616 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:26.892616 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.892616 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:26.893530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:26.894123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:26.894123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.894123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:26.894123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.894123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.894123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:26.894123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.894123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.896103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:26.896501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.896957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.896957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:26.896957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.896957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.899780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:26.900240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.900647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.900647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:26.900647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.900647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:26.918112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:26.918520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.919337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:31:26.919337 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.919337 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:26.919337 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.919337 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.919337 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:26.919337 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.919337 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:26.921958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.922335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:26.922706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:26.923117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.923634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:26.923634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.923634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.923634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:26.923634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.923634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:26.932184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:26.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:26.932867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:26.933046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:26.933612 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:26.933973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.934673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:26.934673 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.934673 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:26.934673 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.934673 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.934673 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:26.934673 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.934673 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:26.935987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:26.936764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:26.936764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.936764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:26.936764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.936764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.936764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:26.936764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.936764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:26.937919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.938722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:26.939330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:26.939985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:26.940887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:26.940887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:26.940887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:26.940887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:26.940887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:26.940887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:26.940887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:26.940887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:27.032844 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:27.034114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:27.034114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:27.034114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:27.034393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:27.034920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:27.035190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:27.036016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:27.036016 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:27.036016 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:27.036016 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:27.036016 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:27.036016 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:27.036016 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:27.036016 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:27.037057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:27.037536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:27.037967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:27.038317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:27.038797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:27.039349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:27.039349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:31.859370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:31.860126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:31.860126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:31.861011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:31.872423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:31.873013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:31.873477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:31.875684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:31.876905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.877961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:31.877961 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.877961 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:31.877961 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.877961 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.877961 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:31.877961 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.877961 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:31.884703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.885381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:31.885969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.885969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.887692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.888580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:31.889443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.889443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:31.889443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:31.889443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:31.889443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:31.889443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:31.890673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:31.891913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.892806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:31.892806 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.892806 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:31.892806 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.892806 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.892806 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:31.892806 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.892806 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:31.895335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:31.895749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:31.895749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.895749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:31.895749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.895749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.895749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:31.895749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.895749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.897120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:31.897473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.897810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.897810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:31.897810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.897810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.898073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:31.898511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.899079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.899079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:31.899079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.899079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:31.914760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:31.915085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.916293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Traceback (most recent call last):
Apr 21 13:31:31.916293 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.916293 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:31.916293 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.916293 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.916293 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:31.916293 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.916293 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:31.917570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.918131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:31.918735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.918735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:31.918897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.919642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.920506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:31.920506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.920506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:31.932706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:31.933145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:31.933239 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:31.933390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:31.933948 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:31.934401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.935241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:31.935241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.935241 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:31.935241 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.935241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.935241 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:31.935241 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.935241 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:31.936507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:31.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:31.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:31.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:31.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.937436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:31.937832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.938572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.939254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:31.939254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.939254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:31.939404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:31.940162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:31.941166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:31.941166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:31.941166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:31.941166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:32.033699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:32.034295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:32.034492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:32.034814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:32.035516 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:32.035909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:32.036095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:32.037051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:32.037051 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:32.037051 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:32.037051 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:32.037051 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:32.037051 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:32.037051 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:32.037051 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:32.038982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:32.039898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:32.041533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:32.042430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:32.042950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:32.042950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:32.042950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:36.859651 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:36.860546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:36.860546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:36.862002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:36.864063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:31:36.872839 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:36.873944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:36.873944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:36.874558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:36.874852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.876528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:36.876528 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.876528 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:36.876528 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.876528 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.876528 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:36.876528 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.876528 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:36.880334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.881036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.885743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:36.886628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.887598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.887598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:36.887598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.887598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:36.888614 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:36.890134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:36.890371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:36.890549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:36.892082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:36.892320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.893832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:36.893832 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.893832 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:36.893832 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.893832 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.893832 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:36.893832 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.893832 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:36.895555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:36.896038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:36.896038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.896038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:36.896038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.896038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.896038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:36.896038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.896038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.896421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:36.896787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.897128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.897128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:36.897128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.897128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.900047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:36.900592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.901380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.901380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:36.901380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.901380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:36.908321 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:36.908559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.909589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:31:36.909589 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.909589 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:36.909589 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.909589 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.909589 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:36.909589 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.909589 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:36.910295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.910679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:36.911146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.911146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:36.911372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.911845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.912509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:36.912509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.912509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:36.933572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:36.934163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:36.934379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:36.934622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:36.935253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:36.935614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.936455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:31:36.936455 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.936455 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:36.936455 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.936455 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.936455 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:36.936455 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.936455 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:36.937995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:36.938732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:36.938732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:36.938732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.938732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:36.938732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.938732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.938732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:36.938732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.938732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:36.939647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.939647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:36.939647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:36.939647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:36.939647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.939647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:36.939647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:36.939647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:36.939647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:36.939647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.939647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:36.940365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.941026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.941026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:36.941026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.941026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:36.941516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:36.942330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:36.942855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:36.942855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:36.942855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:36.942855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:37.034251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:37.034882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:37.035001 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:37.035233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:37.035929 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:37.036549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:37.036755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:37.037986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:37.037986 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:37.037986 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:37.037986 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:37.037986 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:37.037986 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:37.037986 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:37.037986 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:37.039534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:37.040319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:37.040875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:37.041172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:37.042009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:37.043197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:37.043197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:41.860400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:41.861122 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:41.861400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:41.864031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:41.874175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:41.875027 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:41.875426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:41.877722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:41.878082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.879337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:41.879337 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.879337 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:41.879337 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.879337 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.879337 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:41.879337 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.879337 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:41.883790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.884382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:41.885884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.885884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.887735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.888608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:41.889665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.889665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:41.890490 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:41.893712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:41.894743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:41.895006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:41.896696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:41.896953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.897689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:41.897689 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.897689 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:41.897689 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.897689 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.897689 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:41.897689 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.897689 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:41.899500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:41.899951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:41.899951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.899951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:41.899951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.899951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.899951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:41.899951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.899951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.901857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:41.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.903125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.903125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:41.903125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.903125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.904921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:41.905726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.906757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.906757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:41.906757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.906757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:41.920420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:41.920729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:31:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.060 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:41.922766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.923313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:41.923917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.923917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:41.924021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.924796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.925803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:41.925803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.925803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:41.934071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:41.934524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:41.934621 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:41.934740 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:41.936499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:41.936741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.937416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:41.937416 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.937416 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:41.937416 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.937416 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.937416 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:41.937416 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.937416 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:41.938746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:41.939819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:41.939819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.939819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:41.939819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.939819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.939819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:41.939819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.939819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:41.942119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.942741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.943412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:41.943412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.943412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:41.945389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:41.945949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:41.946505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:41.946505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:41.946505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:41.946505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:42.035007 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:42.035436 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:42.035615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:42.035863 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:42.036569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:42.036971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:42.037163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:42.038005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:42.038005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:42.038005 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:42.038005 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:42.038005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:42.038005 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:42.038005 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:42.038005 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:42.039645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:42.040453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:42.041185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:42.041481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:42.042549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:42.043934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:42.043934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:46.861307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:46.862042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:46.862042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:46.863647 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:46.874798 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:46.874985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:46.875059 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:46.876078 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:46.876534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.879947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:46.879947 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.879947 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:46.879947 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.879947 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.879947 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:46.879947 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.879947 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:46.880964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.881583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:46.882243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.882243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.882954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.883882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:46.885107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.885107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:46.893704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:46.895579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:46.895667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:46.895724 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:46.896409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:46.900092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.900092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:46.900092 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.900092 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:46.900092 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.900092 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.900092 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:46.900092 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.900092 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:46.900584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:46.901166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:46.901166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.901166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:46.901166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.901166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.901166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:46.901166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.901166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.902174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:46.903065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:46.903681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:46.904232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.904843 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:46.905335 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.905335 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.905335 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:46.905335 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.905335 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:46.905894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.906380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:46.906861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.906861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:46.907181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.907617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.908091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:46.908091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.908091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:46.934651 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:46.936550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:46.936641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:46.936712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:46.937297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:46.937566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.938140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:31:46.938140 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.938140 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:46.938140 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.938140 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.938140 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:46.938140 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.938140 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:46.938781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:46.939451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:46.940095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:46.940667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:46.941018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:46.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:47.036097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:47.036451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:47.036555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:47.036762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:47.037503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:47.038004 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:47.038198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:47.040505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:47.040505 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:47.040505 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:47.040505 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:47.040505 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:47.040505 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:47.040505 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:47.040505 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:47.041472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:47.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:47.043133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:47.044241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:47.046285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:47.046285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:47.046285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:51.862045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:51.862850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:51.862850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:51.864595 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:51.875483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:51.875798 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:51.875873 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:51.876666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:51.877007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.877897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:51.877897 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.877897 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:51.877897 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.877897 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.877897 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:51.877897 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.877897 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:51.879831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.880649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:51.881321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.881321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.882469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.883338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:51.884295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.884295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:51.895646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:51.896183 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:51.896342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:51.897264 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:51.899247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:51.899461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.900151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:51.900151 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.900151 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:51.900151 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.900151 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.900151 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:51.900151 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.900151 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:51.903953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:51.904761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:51.904761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.904761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:51.904761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.904761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.904761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:51.904761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.904761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.907134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:51.907755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.908185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.908185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:51.908185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.908185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.909686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:51.910182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.910749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.910749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:51.910749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.910749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:51.912930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:51.913456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.914317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:31:51.914317 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.914317 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:51.914317 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.914317 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.914317 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:51.914317 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.914317 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:51.915809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.916253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:51.916573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.916573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:51.917435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.917848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.918350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:51.918350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.918350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:51.936579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:51.937181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:51.937430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:51.937648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:51.938235 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:51.938650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.939486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:31:51.939486 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.939486 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:51.939486 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.939486 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.939486 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:51.939486 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.939486 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:51.940863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:51.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:51.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:51.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:51.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:51.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:51.942362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.942362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:51.942362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:51.942362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:51.942362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.942362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:51.942362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:51.942362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:51.942362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:51.942362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.942362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:51.942897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.943491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.943491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:51.943491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.943491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:51.944148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:51.944852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:51.945354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:51.945354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:51.945354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:51.945354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:52.036780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:52.037188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:52.037369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:52.037586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:52.038338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:52.038750 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:52.038992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:52.039985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:52.039985 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:52.039985 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:52.039985 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:52.039985 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:52.039985 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:52.039985 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:52.039985 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:52.041420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:52.041799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:52.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:52.043038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:52.043619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:52.044137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:52.044137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:56.862729 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:56.863557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:56.863557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:56.864693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:56.875986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:56.876881 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:56.877046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:56.878502 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:56.879192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.881813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:56.881813 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.881813 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:56.881813 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.881813 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.881813 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:56.881813 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.881813 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:56.884054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.884590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:56.885143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.885143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.886666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.887402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:56.888224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.888224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:56.896691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:56.897163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:56.897349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:56.897510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:31:56.898076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:56.898962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.899727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:56.899727 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.899727 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:56.899727 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.899727 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.899727 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:56.899727 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.899727 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:56.903710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:56.904453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:56.904453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.904453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:56.904453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.904453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.904453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:56.904453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.904453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.906197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:56.906742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.907395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:56.908033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:56.908866 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.909632 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.909632 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:56.909632 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.909632 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:56.909849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.910407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:56.910968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:56.911635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.912509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:56.912509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.912509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.912509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:56.912509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.912509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:56.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:56.937460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:56.937655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:56.937926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:56.938515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:56.938876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.939698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:31:56.939698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.939698 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:56.939698 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.939698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.939698 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:56.939698 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.939698 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:31:56.940924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:31:56.941653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:56.941653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:31:56.941653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.941653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:31:56.941653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.941653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.941653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:31:56.941653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.941653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:31:56.942347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.942347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:56.942347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:56.942347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:56.942347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.942347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:56.942347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:56.942347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:56.942347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:56.942347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.942347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:56.942854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.943574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.943574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:56.943574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.943574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:56.944065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:56.944913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:56.945765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:56.945765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:56.945765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:56.945765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:31:57.037464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:31:57.037939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:31:57.038088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:31:57.038358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:31:57.039020 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:57.039469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:31:57.039659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:57.040479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:31:57.040479 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:57.040479 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:31:57.040479 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:31:57.040479 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:57.040479 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:31:57.040479 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:57.040479 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:57.041992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:31:57.042673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:31:57.043288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:31:57.044053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:31:57.044904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:31:57.044904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:31:57.044904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:01.863511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:01.865056 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:01.865056 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:01.866248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:01.876903 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:01.878093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:01.880532 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:01.883047 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:01.883537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.885397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:01.885397 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.885397 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:01.885397 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.885397 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.885397 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:01.885397 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.885397 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:01.888402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.888739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:01.889080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.889080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.891065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.891559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:01.892040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.892040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:01.897397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:01.899745 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:01.899907 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:01.900067 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:01.901860 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:01.903063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.904720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:01.904720 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.904720 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:01.904720 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.904720 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.904720 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:01.904720 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.904720 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:01.906165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:01.906767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:01.906767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.906767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:01.906767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.906767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.906767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:01.906767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.906767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.907993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:01.908388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.908832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.908832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:01.908832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.908832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:01.908832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.909622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:01.910179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.911115 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:01.911980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.912328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:01.912743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.912743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:01.913259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.913783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.914416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:01.914416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.914416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:01.937710 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:01.938129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:01.938342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:01.938598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:01.939116 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:01.939506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.940278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:01.940278 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.940278 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:01.940278 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.940278 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.940278 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:01.940278 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.940278 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:01.941628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:01.942022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:01.942022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:01.942022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.942022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:01.942022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.942022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.942022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:01.942022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.942022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:01.943078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.943078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:01.943078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:01.943078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:01.943078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.943078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:01.943078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:01.943078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:01.943078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:01.943078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.943078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:01.943408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.943763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.943763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:01.943763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.943763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:01.944803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:01.945255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:01.945742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:01.945742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:01.945742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:01.945742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:02.038081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:02.038730 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:02.038871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:02.039120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:02.039796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:02.040243 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:02.040452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:02.041556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:02.041556 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:02.041556 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:02.041556 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:02.041556 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:02.041556 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:02.041556 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:02.041556 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:02.043186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:02.043659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:02.044045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:02.044977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:02.045537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:02.045995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:02.045995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:06.865017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:06.865882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:06.866490 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:06.867915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:06.878441 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:06.879023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:06.879352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:06.880733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:06.882192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.883564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:06.883564 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.883564 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:06.883564 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.883564 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.883564 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:06.883564 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.883564 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:06.888604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.889317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:06.889940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.889940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.891715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.892389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:06.892854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.892854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:06.901179 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:06.901732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:06.902023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:06.902155 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:06.905283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:06.906131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.906864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:06.906864 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.906864 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:06.906864 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.906864 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.906864 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:06.906864 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.906864 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:06.908650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:06.909098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:06.909098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.909098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:06.909098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.909098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.909098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:06.909098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.909098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.909676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:06.910456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.911256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.911256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:06.911256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.911256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.911697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:06.912135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.912678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.912678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:06.912678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.912678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:06.919699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:06.920125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.921124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:32:06.921124 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.921124 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:06.921124 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.921124 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.921124 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:06.921124 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.921124 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:06.922470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.922800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:06.923140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.923140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:06.923572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.923972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.924746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:06.924746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.924746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:06.938328 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:06.938859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:06.939062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:06.939386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:06.940015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:06.940447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.941488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:06.941488 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.941488 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:06.941488 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.941488 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.941488 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:06.941488 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.941488 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:06.943091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:06.943818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:06.943818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:06.943818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.943818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:06.943818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.943818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.943818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:06.943818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.943818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:06.944694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.944694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:06.944694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:06.944694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:06.944694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.944694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:06.944694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:06.944694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:06.944694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:06.944694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.944694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:06.945573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.946101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.946101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:06.946101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.946101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:06.946655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:06.948181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:06.949020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:06.949020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:06.949020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:06.949020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:07.038819 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:07.039486 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:07.039631 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:07.039857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:07.040785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:07.041367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:07.041573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:07.042533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:07.042533 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:07.042533 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:07.042533 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:07.042533 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:07.042533 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:07.042533 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:07.042533 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:07.043943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:07.044566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:07.045167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:07.045638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:07.046354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:07.047421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:07.047421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:11.865624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:11.866496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:11.866496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:11.867791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:11.880759 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:11.881059 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:11.881185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:11.889165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:11.889701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.891001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:11.891001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.891001 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:11.891001 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.891001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.891001 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:11.891001 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.891001 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:11.892201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.892718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:11.893272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.893272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.895975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.897390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:11.898314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.898314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:11.901657 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:11.902548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:11.903802 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:11.903982 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:11.906878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:11.907823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.912547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:11.912547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.912547 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:11.912547 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.912547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.912547 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:11.912547 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.912547 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:11.914330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:11.915070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:11.915070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.915070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:11.915070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.915070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.915070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:11.915070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.915070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.918996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:11.919938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.921175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.921175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:11.921175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.921175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.921939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:11.922456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.922971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.922971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:11.922971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.922971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:11.922971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:11.922971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.923436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:32:11.923436 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.923436 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:11.923436 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.923436 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.923436 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:11.923436 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.923436 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:11.924511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.924848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:11.925179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.925179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:11.925788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.926315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.926773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:11.926773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.926773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:11.938846 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:11.939180 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:11.939500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:11.939725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:11.940238 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:11.940693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:11.941588 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.941588 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:11.941588 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.941588 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.941588 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:11.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.941588 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:11.942786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:11.943477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:11.943477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:11.943477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.943477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:11.943477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.943477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.943477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:11.943477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.943477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:11.944227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.944227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:11.944227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:11.944227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:11.944227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.944227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:11.944227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:11.944227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:11.944227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:11.944227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.944227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:11.944753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.945541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.945541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:11.945541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.945541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:11.946411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:11.947111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:11.947679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:11.947679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:11.947679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:11.947679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:12.039622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:12.040256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:12.040360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:12.040599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:12.041347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:12.041822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:12.042047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:12.042971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:12.042971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:12.042971 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:12.042971 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:12.042971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:12.042971 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:12.042971 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:12.042971 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:12.044119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:12.045130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:12.045193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:12.045804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:12.046493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:12.046493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:16.866623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:16.867448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:16.867448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:16.868835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:16.882090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:16.883980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:16.883980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:16.884535 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:16.884812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.885435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:16.885435 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.885435 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:16.885435 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.885435 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.885435 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:16.885435 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.885435 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:16.888139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.890049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:16.890817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.890817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.892464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.893357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:16.894439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.894439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:16.903533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:16.906544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:16.906722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:16.906927 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:16.908688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:16.909757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.910750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:16.910750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.910750 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:16.910750 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.910750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.910750 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:16.910750 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.910750 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:16.912281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:16.912718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:16.912718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.912718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:16.912718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.912718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.912718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:16.912718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.912718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:16.912718 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.913379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:16.914008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.914923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.914923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:16.914923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.914923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:16.914923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.915185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:16.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:16.917058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:16.917968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:16.918635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:16.919203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.920163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:16.939523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:16.939965 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:16.940062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:16.940224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:16.941174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:16.941889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.942939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:16.942939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.942939 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:16.942939 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.942939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.942939 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:16.942939 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.942939 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:16.943830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:16.944811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:16.945713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:16.946471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:16.946919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:16.947630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:17.040445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:17.041111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:17.041612 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:17.041795 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:17.042579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:17.043002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:17.043282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:17.044132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:17.044132 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:17.044132 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:17.044132 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:17.044132 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:17.044132 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:17.044132 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:17.044132 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:17.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:17.046956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:17.047602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:17.047797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:17.048484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:17.049182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:17.049182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:21.867394 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:21.868318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:21.868318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:21.869627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:21.885193 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:21.886531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:21.886730 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:21.888562 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:21.888905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.890879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:21.890879 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.890879 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:21.890879 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.890879 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.890879 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:21.890879 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.890879 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:21.893182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.893790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:21.894344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.894344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.895244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.895955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:21.896903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.896903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:21.905881 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:21.906428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:21.906527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:21.906606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:21.907125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:21.907383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.909433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:21.909433 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.909433 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:21.909433 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.909433 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.909433 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:21.909433 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.909433 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:21.911080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:21.911953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:21.911953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.911953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:21.911953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.911953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.911953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:21.911953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.911953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.913035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:21.913726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.914356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.914356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:21.914356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.914356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.914598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:21.915550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Traceback (most recent call last):
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.916610 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:21.917369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.917933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:21.918493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:21.919156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.919992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:21.919992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.919992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.919992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:21.919992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.919992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:21.940090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:21.940773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:21.940884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:21.941087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:21.942000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:21.942437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.943337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:21.943337 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.943337 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:21.943337 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.943337 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.943337 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:21.943337 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.943337 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:21.944483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:21.945167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:21.945167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:21.945167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.945167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:21.945167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.945167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.945167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:21.945167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.945167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:21.945574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.945574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:21.945574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:21.945574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:21.945574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.945574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:21.945574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:21.945574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:21.945574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:21.945574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.945574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:21.946153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:21.947487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:21.948332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:21.948332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:21.948332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:21.948332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:21.948332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:21.948332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:21.948332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:21.948332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:21.948332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:21.948332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:21.948332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:22.041087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:22.041692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:22.041832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:22.042162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:22.043044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:22.043530 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:22.043730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:22.044696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:22.044696 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:22.044696 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:22.044696 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:22.044696 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:22.044696 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:22.044696 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:22.044696 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:22.046434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:22.046969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:22.047526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:22.048520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:22.049347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:22.050107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:22.050107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:26.868692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:26.869444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:26.869500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:26.870794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:26.887045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:26.888289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:26.888373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:26.889498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:26.890532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.892082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:26.892082 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.892082 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:26.892082 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.892082 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.892082 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:26.892082 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.892082 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:26.893420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.893988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:26.894545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.894545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.895967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.896415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:26.896890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.896890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:26.907046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:26.907791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:26.908038 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:26.908336 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:26.909378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:26.909680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.910420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:32:26.910420 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.910420 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:26.910420 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.910420 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.910420 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:26.910420 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.910420 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:26.911502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.912404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:26.913164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:26.913864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.914365 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:26.914868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:26.915303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:26.915303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.915303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:26.915303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.915303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.915303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:26.915303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.915303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.915726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:26.916064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.916429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.916429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:26.916429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.916429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.916905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:26.917402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.917907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.917907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:26.917907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.917907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:26.940810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:26.942301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:26.942301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:26.942301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:26.942752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:26.943058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.943785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:26.943785 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.943785 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:26.943785 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.943785 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.943785 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:26.943785 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.943785 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:26.945369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:26.946362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:26.946362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:26.946362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.946362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:26.946362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.946362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.946362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:26.946362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.946362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:26.947066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.947066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:26.947066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:26.947066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:26.947066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.947066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:26.947066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:26.947066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:26.947066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:26.947066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.947066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:26.947601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.948175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.948175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:26.948175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.948175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:26.949250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:26.950284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:26.951103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:26.951103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:26.951103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:26.951103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:27.041835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:27.042425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:27.042593 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:27.042815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:27.043523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:27.043963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:27.044145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:27.045041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:27.045041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:27.045041 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:27.045041 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:27.045041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:27.045041 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:27.045041 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:27.045041 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:27.046574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:27.047302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:27.047874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:27.048156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:27.048950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:27.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:27.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:31.869892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:31.871408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:31.871408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:31.872250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:31.888998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:31.890088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:31.890247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:31.892140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:31.892573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.894479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:31.894479 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.894479 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:31.894479 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.894479 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.894479 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:31.894479 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.894479 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:31.896841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.897543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:31.898842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.898842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.898967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.899809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:31.900449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.900449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:31.909367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:31.909997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:31.910415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:31.911819 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:31.913170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:31.913585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.915241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:32:31.915241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.915241 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:31.915241 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.915241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.915241 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:31.915241 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.915241 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:31.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:31.918447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:31.918447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:31.918447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.918447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:31.918447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.918447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.918447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:31.918447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.918447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:31.918447 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.920144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:31.920541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.920919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.920919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:31.920919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.920919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:31.920919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:31.923587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:31.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.924553 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:31.925669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.926033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:31.926425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.926425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:31.927230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.927748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.928307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:31.928307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.928307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:31.941690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:31.942245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:31.942510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:31.942730 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:31.943290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:31.943793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.944802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:31.944802 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.944802 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:31.944802 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.944802 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.944802 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:31.944802 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.944802 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:31.946179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:31.946974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:31.946974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:31.946974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.946974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:31.946974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.946974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.946974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:31.946974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.946974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:31.947673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.947673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:31.947673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:31.947673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:31.947673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.947673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:31.947673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:31.947673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:31.947673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:31.947673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.947673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:31.948203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.948850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.948850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:31.948850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.948850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:31.949389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:31.950274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:31.951095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:31.951095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:31.951095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:31.951095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:32.042605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:32.043266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:32.043457 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:32.043764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:32.044552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:32.044972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:32.045170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:32.046109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:32.046109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:32.046109 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:32.046109 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:32.046109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:32.046109 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:32.046109 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:32.046109 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:32.047775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:32.048482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:32.049091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:32.049560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:32.050506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:32.051372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:32.051372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:36.870598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:36.871427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:36.871427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:36.872605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:36.890249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:36.890603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:36.891484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:36.894069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:36.894509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.895543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:36.895543 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.895543 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:36.895543 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.895543 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.895543 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:36.895543 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.895543 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:36.898042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.898556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:36.899042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.899042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.899922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.900648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:36.901619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.901619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:36.903661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=251 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:32:36.910536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:36.911166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:36.911424 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:36.912584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:36.915024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:36.915570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.917836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:36.917836 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.917836 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:36.917836 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.917836 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.917836 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:36.917836 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.917836 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:36.921931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:36.922814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:36.922814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.922814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:36.922814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.922814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.922814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:36.922814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.922814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.927243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:36.927795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.928395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.928395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:36.928395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.928395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.930391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:36.931166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.931983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.931983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:36.931983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.931983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:36.932243 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:36.932523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.933150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:32:36.933150 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.933150 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:36.933150 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.933150 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.933150 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:36.933150 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.933150 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task check_fdb_aging_settings failed after 0.062 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:36.934485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.935008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:36.935383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.935383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:36.935646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.936062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.936563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:36.936563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.936563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:36.942427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:36.943520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:36.943717 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:36.943969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:36.944982 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:36.945471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.946642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:36.946642 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.946642 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:36.946642 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.946642 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.946642 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:36.946642 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.946642 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:36.948661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:36.949411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:36.949411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:36.949411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.949411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:36.949411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.949411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.949411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:36.949411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.949411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:36.950140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.950140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:36.950140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:36.950140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:36.950140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.950140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:36.950140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:36.950140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:36.950140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:36.950140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.950140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:36.950684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.951417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.951417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:36.951417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.951417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:36.952013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:36.952471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:36.952940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:36.952940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:36.952940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:36.952940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:37.043430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:37.044026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:37.044196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:37.044587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:37.045683 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:37.046170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:37.046504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:37.047564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:37.047564 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:37.047564 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:37.047564 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:37.047564 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:37.047564 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:37.047564 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:37.047564 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:37.049441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:37.050016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:37.050600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:37.051511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:37.052273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:37.053200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:37.053200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:41.871553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:41.872519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:41.872519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:41.873766 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:41.891417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:41.892955 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:41.894085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:41.896448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:41.897900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.899979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:41.899979 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.899979 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:41.899979 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.899979 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.899979 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:41.899979 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.899979 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:41.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.904049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:41.904874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.904874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.906077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.906620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:41.907263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.907263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:41.911075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:41.911527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:41.911740 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:41.911879 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:41.914753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:41.914971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.915788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:41.915788 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.915788 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:41.915788 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.915788 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.915788 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:41.915788 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.915788 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:41.918233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:41.919137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:41.919137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.919137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:41.919137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.919137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.919137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:41.919137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.919137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.919637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:41.920290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.921081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.921081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:41.921081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.921081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.921594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:41.922087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.922653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.922653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:41.922653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.922653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:41.925980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:41.926304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.927011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 13:32:41.927011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.927011 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:41.927011 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.927011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.927011 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:41.927011 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.927011 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:41.928354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.928722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:41.929141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.929141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:41.929287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.930457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:41.930457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.930457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:41.943320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:41.943861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:41.944370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:41.945434 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:41.945986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:41.946293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.947251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:41.947251 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.947251 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:41.947251 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.947251 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.947251 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:41.947251 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.947251 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:41.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:41.948964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:41.949725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:41.950504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:41.951137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:41.952670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:42.044276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:42.044803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:42.044959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:42.045347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:42.046143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:42.046891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:42.047117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:42.048118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:42.048118 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:42.048118 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:42.048118 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:42.048118 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:42.048118 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:42.048118 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:42.048118 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:42.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:42.050464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:42.050842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:42.051931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:42.052474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:42.053247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:42.053247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:46.872139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:46.873005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:46.873005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:46.874172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:46.893398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:46.894996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:46.895253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:46.899700 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:46.900148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.903733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:46.903733 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.903733 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:46.903733 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.903733 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.903733 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:46.903733 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.903733 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:46.905586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.906006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:46.906485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.906485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.909115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.909922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:46.910851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.910851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:46.912293 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:46.914775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:46.917281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:46.917446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:46.919034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:46.919354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.920969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:46.920969 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.920969 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:46.920969 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.920969 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.920969 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:46.920969 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.920969 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:46.922981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:46.923388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:46.923388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.923388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:46.923388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.923388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.923388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:46.923388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.923388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.924113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:46.924572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.924989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.924989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:46.924989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.924989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.926088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:46.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.927395 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:46.928797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.929385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:46.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.929834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:46.930881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.931470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.932191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:46.932191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.932191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:46.943923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:46.944577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:46.945087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:46.945367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:46.945902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:46.946293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.946954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:46.946954 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.946954 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:46.946954 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.946954 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.946954 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:46.946954 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.946954 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:46.947892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:46.948589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:46.948589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:46.948589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.948589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:46.948589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.948589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.948589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:46.948589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.948589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:46.949149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.949149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:46.949149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:46.949149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:46.949149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.949149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:46.949149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:46.949149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:46.949149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:46.949149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.949149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:46.949669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:46.950349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:46.950939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:46.951854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:46.951854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:46.951854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:46.951854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:46.951854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:46.951854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:46.951854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:46.951854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:46.951854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:46.951854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:46.951854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:47.045273 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:47.045871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:47.046025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:47.046298 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:47.046984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:47.047459 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:47.047752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:47.048735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:47.048735 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:47.048735 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:47.048735 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:47.048735 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:47.048735 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:47.048735 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:47.048735 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:47.050144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:47.050628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:47.051074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:47.051963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:47.052618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:47.053228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:47.053228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:51.873187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:51.874093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:51.874093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:51.875447 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:51.895065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:51.896348 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:51.896483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:51.898982 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:51.899669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.901356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:51.901356 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.901356 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:51.901356 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.901356 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.901356 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:51.901356 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.901356 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:51.903354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.903765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:51.904100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.904100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.908369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.908787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:51.909146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.909146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:51.915081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:51.915475 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:51.915590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:51.915737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:51.916334 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:51.916955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.919538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:32:51.919538 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.919538 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:51.919538 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.919538 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.919538 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:51.919538 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.919538 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:51.921613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:51.922387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:51.922387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:51.922387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.922387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:51.922387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.922387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.922387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:51.922387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.922387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.922887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:51.923641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:51.924500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:51.925524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.926162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.926705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:51.927164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.927646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.927646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:51.927646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.927646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:51.927887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.928504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.929040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:51.929040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.929040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:51.944409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:51.944884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:51.945062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:51.945313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:51.945874 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:51.946291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.947118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:51.947118 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.947118 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:51.947118 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.947118 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.947118 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:51.947118 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.947118 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:51.948418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:51.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:51.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:51.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:51.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:51.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:51.949914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.949914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:51.949914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:51.949914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:51.949914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.949914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:51.949914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:51.949914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:51.949914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:51.949914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.949914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:51.950233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.950603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.950603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:51.950603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.950603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:51.951684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:51.952041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:51.952402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:51.952402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:51.952402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:51.952402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:52.046024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:52.046462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:52.046859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:52.047205 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:52.047897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:52.048332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:52.048629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:52.049481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:52.049481 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:52.049481 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:52.049481 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:52.049481 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:52.049481 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:52.049481 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:52.049481 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:52.051163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:52.052073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:52.053050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:52.053491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:52.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:52.054874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:52.054874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:56.873932 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:56.874831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:56.874831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:56.876320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:56.897046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:56.898423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:56.898739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:56.901059 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:56.901473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.902900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:56.902900 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.902900 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:56.902900 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.902900 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.902900 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:56.902900 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.902900 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:56.905326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.906281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:56.907410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.907410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.907796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.908390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:56.908949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.908949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:56.916350 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:56.916971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:56.917288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:56.917505 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:32:56.919888 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:56.920480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.921481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:32:56.921481 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.921481 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:56.921481 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.921481 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.921481 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:56.921481 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.921481 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:56.923793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:56.924867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:56.924867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:56.924867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.924867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:56.924867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.924867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.924867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:56.924867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.924867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:56.924867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.926064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:56.926881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.927894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.927894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:56.927894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.927894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:56.927894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:56.929194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:56.930148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.931322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:56.932646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:56.933677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.934274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.935050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:56.935050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.935050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:56.945173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:56.945784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:56.946065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:56.946464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:56.946894 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:56.947186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.947805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:32:56.947805 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.947805 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:56.947805 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.947805 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.947805 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:56.947805 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.947805 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:32:56.948947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:32:56.949696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:56.949696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:32:56.949696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.949696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:32:56.949696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.949696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.949696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:32:56.949696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.949696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:32:56.950245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.950245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:56.950245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:56.950245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:56.950245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.950245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:56.950245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:56.950245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:56.950245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:56.950245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.950245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:56.950772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.951482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.951482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:56.951482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.951482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:56.951665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:56.952559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:56.953140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:56.953140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:56.953140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:56.953140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:32:57.046722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:32:57.047193 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:32:57.047378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:32:57.047623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:32:57.048432 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:57.048868 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:32:57.049080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:57.049966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:32:57.049966 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:57.049966 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:32:57.049966 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:32:57.049966 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:57.049966 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:32:57.049966 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:57.049966 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:57.051322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:32:57.051924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:57.052626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:32:57.053021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:32:57.053765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:32:57.054566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:32:57.054566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:01.874774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:01.875551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:01.875551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:01.876668 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:01.898891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:01.899520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:01.899670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:01.900814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:01.901092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.902817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:01.902817 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.902817 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:01.902817 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.902817 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.902817 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:01.902817 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.902817 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:01.906112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.906800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:01.907376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.907376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.908760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.910556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:01.911899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.911899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:01.916806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:01.917048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:01.917141 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:01.917281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:01.918993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:01.919925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.920748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:33:01.920748 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.920748 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:01.920748 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.920748 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.920748 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:01.920748 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.920748 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:01.921691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:01.922330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:01.922330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:01.922330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.922330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:01.922330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.922330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.922330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:01.922330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.922330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.922914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:01.923473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:01.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:01.924047 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:01.924306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:01.924954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.925677 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:01.926484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.927038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:01.927577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:01.928064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.928599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:01.928599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.928599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.928599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:01.928599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.928599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:01.945691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:01.946309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:01.946579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:01.946833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:01.947607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:01.948117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.949073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:33:01.949073 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.949073 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:01.949073 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.949073 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.949073 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:01.949073 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.949073 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:01.950393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:01.951025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:01.951025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:01.951025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.951025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:01.951025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.951025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.951025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:01.951025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.951025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:01.951819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.951819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:01.951819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:01.951819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:01.951819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.951819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:01.951819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:01.951819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:01.951819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:01.951819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.951819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:01.952372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.953105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.953105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:01.953105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.953105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:01.954334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:01.955029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:01.955574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:01.955574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:01.955574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:01.955574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:02.047375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:02.047899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:02.047989 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:02.048201 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:02.049014 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:02.049446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:02.049704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:02.050826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:02.050826 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:02.050826 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:02.050826 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:02.050826 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:02.050826 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:02.050826 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:02.050826 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:02.052682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:02.053052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:02.053405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:02.054513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:02.055014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:02.055572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:02.055572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:06.875464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:06.876339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:06.876339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:06.877286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:06.899480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:06.901598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:06.901731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:06.905565 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:06.906651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.908524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:06.908524 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.908524 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:06.908524 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.908524 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.908524 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:06.908524 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.908524 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:06.909903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.910552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:06.911042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.911042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.912599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.913144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:06.913817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.913817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:06.918069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:06.919456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:06.919759 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:06.920037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:06.921198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:06.921808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.922817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Traceback (most recent call last):
Apr 21 13:33:06.922817 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.922817 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:06.922817 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.922817 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.922817 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:06.922817 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.922817 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:06.923829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.924505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:06.925028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.925028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:06.925028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:06.925175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.925898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.926753 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:06.927340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:06.928003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:06.929288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.930024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:06.930487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:06.930965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:06.931509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.931509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:06.931509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.931509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.931509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:06.931509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.931509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:06.946190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:06.946619 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:06.946846 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:06.947101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:06.947748 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:06.948151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.949240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:33:06.949240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.949240 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:06.949240 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.949240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.949240 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:06.949240 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.949240 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:06.951006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:06.951693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:06.951693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:06.951693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.951693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:06.951693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.951693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.951693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:06.951693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.951693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:06.953049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.953049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:06.953049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:06.953049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:06.953049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.953049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:06.953049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:06.953049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:06.953049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:06.953049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.953049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:06.953623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.954068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.954068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:06.954068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.954068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:06.954717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:06.955258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:06.955740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:06.955740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:06.955740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:06.955740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:07.047989 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:07.048614 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:07.048782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:07.049130 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:07.050197 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:07.050660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:07.050887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:07.051853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:07.051853 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:07.051853 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:07.051853 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:07.051853 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:07.051853 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:07.051853 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:07.051853 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:07.053812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:07.054408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:07.054987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:07.055567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:07.056400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:07.057108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:07.057108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:11.876952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:11.876952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:11.876952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:11.878398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:11.902705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:11.903835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:11.904723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:11.905538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:11.905829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.907722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:11.907722 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.907722 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:11.907722 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.907722 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.907722 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:11.907722 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.907722 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:11.911186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.911621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:11.912010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.912010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.915058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.915609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:11.916202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.916202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:11.919519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:11.919982 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:11.920343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.920941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:33:11.920941 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.920941 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:11.920941 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.920941 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.920941 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:11.920941 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.920941 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:11.922391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.922784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:11.923183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.923183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:11.923183 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:11.923889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.924413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.924929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:11.924929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.924929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:11.924929 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:11.924929 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:11.925087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:11.925647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.926464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Traceback (most recent call last):
Apr 21 13:33:11.926464 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.926464 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:11.926464 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.926464 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.926464 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:11.926464 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.926464 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:11.927770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:11.928279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:11.928279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:11.928279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.928279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:11.928279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.928279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.928279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:11.928279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.928279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.929110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:11.929476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.929961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.929961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:11.929961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.929961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:11.930234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:11.930843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:11.931382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.931382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:11.931382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.931382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.931382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:11.931382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.931382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:11.946805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:11.947318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:11.947481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:11.947691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:11.948786 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:11.949285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.950363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:33:11.950363 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.950363 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:11.950363 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.950363 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.950363 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:11.950363 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.950363 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:11.951513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:11.952380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:11.952380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:11.952380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.952380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:11.952380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.952380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.952380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:11.952380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.952380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:11.952921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.952921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:11.952921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:11.952921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:11.952921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.952921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:11.952921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:11.952921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:11.952921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:11.952921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.952921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:11.953469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.954040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.954040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:11.954040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.954040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:11.954687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:11.955667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:11.956691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:11.956691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:11.956691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:11.956691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:12.048737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:12.049237 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:12.049370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:12.049637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:12.050415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:12.050800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:12.050989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:12.051838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:12.051838 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:12.051838 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:12.051838 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:12.051838 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:12.051838 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:12.051838 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:12.051838 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:12.053438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:12.054092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:12.054661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:12.055190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:12.056067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:12.057006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:12.057006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:16.876913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:16.877579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:16.877668 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:16.879009 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:16.905384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:16.905384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:16.905384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:16.909313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:16.909642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.911816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:16.911816 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.911816 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:16.911816 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.911816 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.911816 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:16.911816 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.911816 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:16.913125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.913519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:16.913906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.913906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.915374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.916045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:16.916597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.916597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:16.918800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:16.919249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.920189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:33:16.920189 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.920189 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:16.920189 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.920189 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.920189 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:16.920189 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.920189 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:16.921375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.921736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:16.922088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.922088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:16.922538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.922989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.923555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:16.923555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.923555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:16.923555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:16.923555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:16.923731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:16.923802 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:16.924647 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:16.924925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.925736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:33:16.925736 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.925736 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:16.925736 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.925736 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.925736 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:16.925736 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.925736 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:16.926622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:16.927107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:16.927107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:16.927107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.927107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:16.927107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.927107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.927107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:16.927107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.927107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.927602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:16.927947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.928328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.928328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:16.928328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.928328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:16.928890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:16.929353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:16.929839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.929839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:16.929839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.929839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.929839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:16.929839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.929839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:16.947546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:16.948116 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:16.948315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:16.948520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:16.949225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:16.949767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.950832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:33:16.950832 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.950832 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:16.950832 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.950832 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.950832 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:16.950832 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.950832 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:16.952058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:16.952696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:16.952696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:16.952696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.952696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:16.952696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.952696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.952696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:16.952696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.952696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:16.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:16.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:16.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:16.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:16.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:16.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:16.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:16.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:16.954035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.954610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.954610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:16.954610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.954610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:16.955374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:16.955837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:16.956334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:16.956334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:16.956334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:16.956334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:17.049539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:17.050186 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:17.050435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:17.050818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:17.051596 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:17.052026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:17.052336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:17.053199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:17.053199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:17.053199 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:17.053199 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:17.053199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:17.053199 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:17.053199 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:17.053199 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:17.054062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:17.054760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:17.055423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:17.056271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:17.057062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:17.057062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:17.057062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:21.878205 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:21.879178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:21.879178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:21.880658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:21.905772 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:21.906947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:21.907751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:21.909597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:21.909944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.912351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:21.912351 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.912351 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:21.912351 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.912351 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.912351 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:21.912351 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.912351 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:21.914669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.915365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:21.915989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.915989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.916744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.917781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:21.918717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.918717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:21.919528 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:21.919974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.920863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:33:21.920863 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.920863 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:21.920863 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.920863 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.920863 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:21.920863 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.920863 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:21.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.922563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:21.923147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:21.924148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.924979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:21.924979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.924979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.924979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:21.924979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.924979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:21.924979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:21.924979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:21.924979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:21.925650 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:21.925650 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:21.925865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.926623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:33:21.926623 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.926623 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:21.926623 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.926623 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.926623 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:21.926623 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.926623 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:21.928055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:21.928523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:21.928523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:21.928523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.928523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:21.928523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.928523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.928523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:21.928523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.928523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.929595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:21.929933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.930324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.930324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:21.930324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.930324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:21.931104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:21.931558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:21.932036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.932036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:21.932036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.932036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.932036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:21.932036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.932036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:21.948160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:21.948788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:21.948969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:21.949184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:21.950565 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:21.951075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.952248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:33:21.952248 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.952248 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:21.952248 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.952248 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.952248 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:21.952248 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.952248 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:21.953788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:21.954477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:21.954477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:21.954477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.954477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:21.954477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.954477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.954477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:21.954477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.954477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:21.955647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.955647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:21.955647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:21.955647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:21.955647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.955647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:21.955647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:21.955647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:21.955647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:21.955647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.955647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:21.955981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.956347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.956347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:21.956347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.956347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:21.957560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:21.958050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:21.958540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:21.958540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:21.958540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:21.958540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:22.050398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:22.051086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:22.051236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:22.051552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:22.052681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:22.053179 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:22.053680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:22.054730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:22.054730 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:22.054730 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:22.054730 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:22.054730 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:22.054730 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:22.054730 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:22.054730 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:22.056421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:22.057045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:22.057744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:22.058016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:22.058620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:22.059107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:22.059107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:26.879134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:26.880082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:26.880082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:26.881732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:26.907489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:26.908404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:26.909669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:26.912021 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:26.912974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.916301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:33:26.916301 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.916301 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:26.916301 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.916301 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.916301 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:26.916301 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.916301 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:26.918648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.919199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:26.919885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.919885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:26.920999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.921742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.922936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:26.922936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.922936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:26.923444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:26.923875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.924997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:26.925508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:33:26.925508 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.925508 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:26.925508 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.925508 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.925508 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:26.925508 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.925508 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:26.926816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.927362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:26.927690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.927690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:26.927690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:26.928382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.928891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.950401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:26.950401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.950401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:26.950401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:26.950401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:26.950401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:26.950401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.950401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:33:26.950401 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.950401 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:26.950401 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:26.950912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.951644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:26.952427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.953065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:26.953711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:26.955369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:26.955967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:26.956648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:26.957032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:26.957440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:26.957749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:26.958674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:27.051441 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:27.052166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:27.052385 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:27.052634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:27.053406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:27.053805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:27.053996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:27.054899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:27.054899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:27.054899 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:27.054899 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:27.054899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:27.054899 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:27.054899 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:27.054899 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:27.056524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:27.057182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:27.057755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:27.058075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:27.058644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:27.059135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:27.059135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:31.880153 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:31.882431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:31.882431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:31.883451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:31.908238 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:31.908878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:31.910763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:31.912127 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:31.912402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.913080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:33:31.913080 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.913080 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:31.913080 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.913080 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.913080 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:31.913080 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.913080 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:31.914802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.915247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:31.915663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.915663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:31.919091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.919748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.920271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:31.920271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.920271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:31.926505 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:31.927653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:31.927794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:31.927949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:31.935116 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:31.935383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:33:31.937826 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.937826 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:31.937826 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.937826 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.937826 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:31.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:31.943684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:31.944596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:31.944596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:31.944596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.944596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:31.944596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.944596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.944596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:31.944596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.944596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task set_fip_distributed_flag failed after 0.016 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.948028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:31.948462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:31.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.948871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:31.951904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:31.952955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:31.954041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.954041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:31.954041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.954041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.954041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:31.954041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.954041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:31.954041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:31.954041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:31.954041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:31.954920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:31.956487 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:31.957310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.959534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:33:31.959534 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.959534 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:31.959534 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.959534 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.959534 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:31.959534 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.959534 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:31.963319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:31.964055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:31.964055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:31.964055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.964055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:31.964055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.964055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.964055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:31.964055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.964055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:31.965960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_ha_failover failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.965960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:31.965960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:31.965960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:31.965960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.965960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:31.965960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:31.965960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:31.965960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:31.965960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.965960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:31.967339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.967760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.967760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:31.967760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.967760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.970109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:31.970755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.971357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.971357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:31.971357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.971357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:31.981146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:31.981573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.982658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:33:31.982658 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.982658 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:31.982658 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.982658 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.982658 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:31.982658 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.982658 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task check_fdb_aging_settings failed after 0.101 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:31.983823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.984102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:31.984367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.984367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:31.984898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:31.985266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:31.985650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:31.985650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:31.985650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:32.052271 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:32.052928 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:32.053031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:32.053223 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:32.054416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:32.054975 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:32.055203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:32.056276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:32.056276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:32.056276 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:32.056276 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:32.056276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:32.056276 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:32.056276 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:32.056276 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:32.057855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:32.058637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:32.059241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:32.059535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:32.060368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:32.061378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:32.061378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:36.881480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:36.882756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:36.882756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:36.883973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:36.909570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:36.912865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=245 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:33:36.913882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:36.914201 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:36.915913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:36.916084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.916855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:33:36.916855 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.916855 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:36.916855 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.916855 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.916855 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:36.916855 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.916855 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:36.919244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.919889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:36.920467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.920467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:36.923076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.924406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.925230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:36.925230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.925230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:36.926420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:36.926788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.927736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:36.928368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:33:36.928368 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.928368 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:36.928368 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.928368 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.928368 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:36.928368 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.928368 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:36.928843 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:36.930011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.930615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:36.930941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.930941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:36.930941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:36.931494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.932010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.932534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:36.932534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.932534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:36.932534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:36.932534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:36.932534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.933551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:33:36.933551 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.933551 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:36.933551 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.933551 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.933551 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:36.933551 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.933551 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:36.934753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:36.935426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:36.935426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:36.935426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.935426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:36.935426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.935426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.935426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:36.935426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.935426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.936318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:36.936659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.937036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.937036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:36.937036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.937036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:36.938463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:36.938919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:36.939410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.939410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:36.939410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.939410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.939410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:36.939410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.939410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:36.954414 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:36.954943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:36.955130 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:36.955322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:36.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:36.956333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.957338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:33:36.957338 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.957338 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:36.957338 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.957338 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.957338 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:36.957338 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.957338 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:36.958819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:36.959518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:36.959518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:36.959518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.959518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:36.959518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.959518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.959518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:36.959518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.959518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:36.960081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.960081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:36.960081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:36.960081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:36.960081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.960081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:36.960081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:36.960081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:36.960081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:36.960081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.960081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:36.960639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:36.961371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:36.961837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:36.962384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:36.962384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:36.962384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:36.962384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:36.962384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:36.962384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:36.962384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:36.962384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:36.962384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:36.962384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:36.962384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:37.052898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:37.053287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:37.053367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:37.053552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:37.054247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:37.054646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:37.054833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:37.055864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:37.055864 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:37.055864 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:37.055864 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:37.055864 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:37.055864 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:37.055864 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:37.055864 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:37.057407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:37.058015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:37.058599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:37.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:37.059809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:37.060597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:37.060597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:41.882327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:41.883163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:41.883163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:41.884488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:41.911700 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:41.913579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:41.913579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:41.916534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:41.916950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.919123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:33:41.919123 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.919123 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:41.919123 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.919123 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.919123 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:41.919123 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.919123 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:41.921063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.921725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:41.922308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.922308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:41.926996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.927501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.928096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:41.928096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.928096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:41.928812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:41.930367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:41.930625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:41.930836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:41.932133 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:41.932512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.933414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:33:41.933414 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.933414 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:41.933414 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.933414 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.933414 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:41.933414 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.933414 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:41.935007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:41.935495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:41.935495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:41.935495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.935495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:41.935495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.935495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.935495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:41.935495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.935495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:41.935495 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.937080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:41.937657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.938276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.938276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:41.938276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.938276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:41.938276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:41.939720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:41.940445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.941199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:41.941964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.942513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:41.943020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.943805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.944323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:41.944323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.944323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:41.954852 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:41.955686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:41.955975 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:41.956311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:41.956897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:41.957279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.958017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:33:41.958017 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.958017 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:41.958017 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.958017 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.958017 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:41.958017 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.958017 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:41.959192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:41.960077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:41.960077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:41.960077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.960077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:41.960077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.960077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.960077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:41.960077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.960077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:41.960544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.960544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:41.960544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:41.960544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:41.960544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.960544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:41.960544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:41.960544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:41.960544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:41.960544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.960544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:41.961166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:41.961940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:41.962708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:41.963776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:41.963776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:41.963776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:41.963776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:41.963776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:41.963776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:41.963776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:41.963776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:41.963776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:41.963776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:41.963776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:42.053634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:42.054108 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:42.054306 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:42.054530 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:42.055016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:42.055469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:42.055777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:42.056652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:42.056652 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:42.056652 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:42.056652 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:42.056652 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:42.056652 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:42.056652 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:42.056652 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:42.058195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:42.059201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:42.060485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:42.060580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:42.061482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:42.062166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:42.062166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:46.883249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:46.884162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:46.884162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:46.884895 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:46.913119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:46.914312 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:46.914437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:46.916278 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:46.917707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.918785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:33:46.918785 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.918785 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:46.918785 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.918785 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.918785 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:46.918785 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.918785 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:46.919877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.920433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:46.920940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.920940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:46.921439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.921909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.922433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:46.922433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.922433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:46.929363 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:46.929774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.930708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:46.931486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:33:46.931486 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.931486 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:46.931486 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.931486 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.931486 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:46.931486 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.931486 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:46.931959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:46.933169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.933550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:46.933875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.933875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:46.933875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:46.934851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.935531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.936390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:46.936390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.936390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:46.936390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:46.936860 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:46.937247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.938058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:33:46.938058 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.938058 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:46.938058 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.938058 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.938058 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:46.938058 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.938058 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:46.939111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:46.939809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:46.939809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:46.939809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.939809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:46.939809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.939809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.939809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:46.939809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.939809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.940226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:46.940756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.941333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.941333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:46.941333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.941333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:46.941705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:46.942269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:46.942734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.942734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:46.942734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.942734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.942734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:46.942734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.942734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:46.955496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:46.956016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:46.956229 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:46.956509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:46.957044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:46.957435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.958081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:33:46.958081 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.958081 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:46.958081 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.958081 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.958081 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:46.958081 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.958081 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:46.959540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:46.960283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:46.960283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:46.960283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.960283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:46.960283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.960283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.960283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:46.960283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.960283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:46.961059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.961059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:46.961059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:46.961059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:46.961059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.961059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:46.961059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:46.961059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:46.961059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:46.961059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.961059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:46.961637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.962266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.962266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:46.962266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.962266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:46.963184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:46.964021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:46.964917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:46.964917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:46.964917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:46.964917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:47.054359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:47.054931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:47.055076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:47.055350 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:47.055945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:47.056469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:47.056665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:47.057536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:47.057536 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:47.057536 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:47.057536 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:47.057536 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:47.057536 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:47.057536 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:47.057536 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:47.059023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:47.059430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:47.059771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:47.060699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:47.061163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:47.061637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:47.061637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:51.884165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:51.885129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:51.885129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:51.886453 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:51.914529 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:51.914960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:51.915302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:51.917188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:51.917537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.919988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:33:51.919988 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.919988 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:51.919988 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.919988 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.919988 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:51.919988 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.919988 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:51.921919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.922908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:51.923867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.923867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:51.925116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.926003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.926544 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:51.927443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.927784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:51.928106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.928106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:51.928588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.929014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.929536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:51.929536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.929536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:51.931301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:51.931737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:51.931967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:51.932165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:51.932518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:51.932690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.933276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:33:51.933276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.933276 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:51.933276 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.933276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.933276 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:51.933276 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.933276 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:51.934145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:51.934611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:51.934611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:51.934611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.934611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:51.934611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.934611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.934611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:51.934611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.934611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.935304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:51.935668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.936040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.936040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:51.936040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.936040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:51.936643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:51.937168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:51.937671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.937671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:51.937671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.937671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.937671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:51.937671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.937671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:51.956072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:51.956685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:51.956957 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:51.957309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:51.957890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:51.958524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.959621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:33:51.959621 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.959621 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:51.959621 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.959621 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.959621 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:51.959621 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.959621 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:51.961013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:51.961994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:51.961994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:51.961994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.961994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:51.961994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.961994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.961994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:51.961994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.961994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:51.962768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.962768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:51.962768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:51.962768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:51.962768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.962768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:51.962768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:51.962768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:51.962768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:51.962768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.962768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:51.963486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.964359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.964359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:51.964359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.964359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:51.964620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:51.965757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:51.966866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:51.966866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:51.966866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:51.966866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:52.055015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:52.055424 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:52.055620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:52.055935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:52.056494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:52.056786 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:52.056923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:52.057683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:52.057683 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:52.057683 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:52.057683 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:52.057683 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:52.057683 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:52.057683 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:52.057683 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:52.058763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:52.059130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:52.059463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:52.059947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:52.060415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:52.060904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:52.060904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:56.885024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:56.885859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:56.885859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:56.887050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:56.915164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:56.915831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:56.916971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:56.918658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:56.919109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.920576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:56.920969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.921515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:33:56.921515 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.921515 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:56.921515 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.921515 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.921515 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:56.921515 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.921515 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:56.923456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.924182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:56.924769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:56.925481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.926310 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:56.926963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.927703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:56.928326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.928326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:56.928998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.929977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.930986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:56.930986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.930986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:56.931949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:56.932667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:56.932729 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:56.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:33:56.933569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:56.933942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.935840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:33:56.935840 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.935840 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:56.935840 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.935840 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.935840 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:56.935840 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.935840 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:56.936778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:56.937118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:56.937118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:56.937118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.937118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:56.937118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.937118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.937118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:56.937118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.937118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.937408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:56.937704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.937994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.937994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:56.937994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.937994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:56.938147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:56.938491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:56.938932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.938932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:56.938932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.938932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.938932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:56.938932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.938932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:56.956626 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:56.957162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:56.957475 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:56.957719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:56.958476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:56.958864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.959740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:33:56.959740 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.959740 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:56.959740 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.959740 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.959740 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:56.959740 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.959740 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:33:56.960682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:33:56.961135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:56.961135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:33:56.961135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.961135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:33:56.961135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.961135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.961135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:33:56.961135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.961135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:33:56.961591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.961591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:56.961591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:56.961591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:56.961591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.961591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:56.961591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:56.961591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:56.961591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:56.961591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.961591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:56.961954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.962363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.962363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:56.962363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.962363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:56.962720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:56.963260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:56.963867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:56.963867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:56.963867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:56.963867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:33:57.055636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:33:57.056112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:33:57.056255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:33:57.056494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:33:57.056980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:57.057400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:33:57.057586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:57.058346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:33:57.058346 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:57.058346 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:33:57.058346 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:33:57.058346 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:57.058346 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:33:57.058346 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:57.058346 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:57.059645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:33:57.060306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:57.060832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:33:57.061242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:33:57.061982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:33:57.062787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:33:57.062787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:01.885731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:01.886899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:01.886899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:01.888165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:01.916042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:01.917670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:01.918875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:01.920642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:01.920954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.922035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:01.922035 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.922035 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:01.922035 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.922035 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.922035 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:01.922035 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.922035 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:01.923819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.924504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:01.925146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.925146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:01.925146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:01.925694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.926437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.926975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:01.926975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.926975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:01.926975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.927228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:01.927228 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.927228 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:01.927228 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.927228 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.927228 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:01.927228 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.927228 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:01.928887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.929346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:01.929668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.929668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:01.930745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.931186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.931755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:01.931755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.931755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:01.932615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:01.933173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:01.933341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:01.933480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:01.933889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:01.934344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.934774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:34:01.934774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.934774 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:01.934774 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.934774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.934774 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:01.934774 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.934774 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:01.937370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:01.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:01.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:01.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:01.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:01.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.937826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.938973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:01.939669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:01.940320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:01.941009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.942041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:01.957251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:01.957743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:01.957967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:01.958191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:01.958751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:01.959161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.960032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:34:01.960032 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.960032 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:01.960032 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.960032 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.960032 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:01.960032 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.960032 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:01.961230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:01.961899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:01.961899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:01.961899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.961899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:01.961899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.961899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.961899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:01.961899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.961899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:01.962695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.962695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:01.962695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:01.962695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:01.962695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.962695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:01.962695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:01.962695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:01.962695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:01.962695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.962695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:01.963287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.963900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.963900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:01.963900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.963900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:01.964523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:01.965696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:01.966696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:01.966696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:01.966696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:01.966696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:02.056436 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:02.056892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:02.057001 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:02.057247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:02.057716 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:02.058154 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:02.058454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:02.059389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:02.059389 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:02.059389 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:02.059389 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:02.059389 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:02.059389 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:02.059389 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:02.059389 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:02.060697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:02.061442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:02.062066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:02.062372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:02.063122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:02.063936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:02.063936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:06.886594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:06.887583 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:06.887583 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:06.888070 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:06.917783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:06.918304 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:06.918763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:06.922048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:06.922610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.923804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:06.923804 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.923804 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:06.923804 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.923804 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.923804 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:06.923804 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.923804 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:06.926313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.926791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:06.927198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:06.927882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.929052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:06.929052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.929052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.929052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:06.929052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.929052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:06.931488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:06.931886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.932894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:06.932894 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.932894 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:06.932894 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.932894 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.932894 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:06.932894 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.932894 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:06.934222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.934698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:06.935109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.935109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:06.935109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:06.936459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.936949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.937462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:06.937462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.937462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:06.937462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:06.937592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:06.937692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:06.938643 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:06.938994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.939912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:34:06.939912 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.939912 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:06.939912 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.939912 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.939912 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:06.939912 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.939912 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:06.941083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:06.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:06.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:06.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:06.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:06.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.941557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.942189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:06.942568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.942947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.942947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:06.942947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.942947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:06.943488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:06.944419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.944419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:06.944419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.944419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.944419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:06.944419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.944419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:06.957774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:06.958232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:06.958374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:06.958527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:06.959447 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:06.959883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.960951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:34:06.960951 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.960951 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:06.960951 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.960951 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.960951 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:06.960951 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.960951 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:06.962371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:06.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:06.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:06.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:06.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:06.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:06.963594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.963594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:06.963594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:06.963594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:06.963594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.963594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:06.963594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:06.963594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:06.963594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:06.963594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.963594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:06.964142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:06.964885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:06.965509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:06.966005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:06.966005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:06.966005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:06.966005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:06.966005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:06.966005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:06.966005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:06.966005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:06.966005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:06.966005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:06.966005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:07.057482 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:07.058071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:07.058192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:07.058466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:07.059373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:07.059826 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:07.060019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:07.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:07.061012 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:07.061012 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:07.061012 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:07.061012 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:07.061012 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:07.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:07.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:07.062230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:07.063050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:07.063713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:07.064477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:07.065388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:07.065388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:07.065388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:11.887405 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:11.888317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:11.888317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:11.888926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:11.918661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:11.919431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:11.919693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:11.921639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:11.922130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.924091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:11.924091 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.924091 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:11.924091 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.924091 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.924091 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:11.924091 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.924091 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:11.925766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.926607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:11.927533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.927533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:11.928202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.928831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.929596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:11.929596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.929596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:11.929596 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:11.929783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.930381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:11.930381 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.930381 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:11.930381 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.930381 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.930381 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:11.930381 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.930381 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:11.931502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.931934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:11.932488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.932488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:11.932692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.933304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.934010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:11.934010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.934010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:11.936033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:11.936405 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:11.936532 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:11.936662 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:11.937031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:11.937285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.937843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:34:11.937843 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.937843 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:11.937843 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.937843 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.937843 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:11.937843 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.937843 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:11.938627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:11.939050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:11.939050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:11.939050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.939050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:11.939050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.939050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.939050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:11.939050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.939050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.939510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:11.939834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.940184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.940184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:11.940184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.940184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:11.940533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:11.940937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:11.941462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.941462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:11.941462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.941462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.941462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:11.941462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.941462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:11.958370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:11.959552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:11.959760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:11.959916 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:11.960707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:11.961124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.962077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:34:11.962077 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.962077 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:11.962077 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.962077 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.962077 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:11.962077 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.962077 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:11.963757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:11.964416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:11.964416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:11.964416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.964416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:11.964416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.964416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.964416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:11.964416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.964416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:11.965502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.965502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:11.965502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:11.965502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:11.965502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.965502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:11.965502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:11.965502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:11.965502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:11.965502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.965502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:11.966044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.966599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.966599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:11.966599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.966599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:11.967506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:11.968524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:11.969393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:11.969393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:11.969393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:11.969393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:12.058304 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:12.059492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:12.059492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:12.059850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:12.060322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:12.060963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:12.061200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:12.062344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:12.062344 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:12.062344 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:12.062344 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:12.062344 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:12.062344 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:12.062344 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:12.062344 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:12.064556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:12.065247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:12.065745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:12.066562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:12.067480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:12.067480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:16.889254 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:16.890265 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:16.890361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:16.892123 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:16.919618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:16.920385 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:16.920705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:16.922245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:16.922477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.923200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:16.923200 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.923200 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:16.923200 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.923200 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.923200 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:16.923200 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.923200 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:16.926677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.927613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:16.928295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.928295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:16.929437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.929925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.930455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:16.930455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.930455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:16.932446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:16.932668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:16.933200 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.933200 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:16.933200 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.933200 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.933200 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:16.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.933200 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:16.934154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.934504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:16.934804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.934804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:16.935219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.935656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.936103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:16.936103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.936103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:16.936683 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:16.937659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:16.937895 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:16.938117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:16.938530 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:16.938695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.941539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:34:16.941539 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.941539 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:16.941539 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.941539 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.941539 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:16.941539 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.941539 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:16.942940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:16.943376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:16.943376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:16.943376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.943376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:16.943376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.943376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.943376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:16.943376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.943376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.943838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:16.944721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.945472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.945472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:16.945472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.945472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:16.947963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:16.948448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:16.948903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.948903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:16.948903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.948903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.948903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:16.948903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.948903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:16.959174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:16.959658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:16.959953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:16.960313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:16.960751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:16.961041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.961800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:34:16.961800 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.961800 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:16.961800 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.961800 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.961800 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:16.961800 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.961800 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:16.963172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:16.963828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:16.963828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:16.963828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.963828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:16.963828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.963828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.963828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:16.963828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.963828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:16.965242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.965242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:16.965242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:16.965242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:16.965242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.965242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:16.965242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:16.965242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:16.965242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:16.965242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.965242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:16.965794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.966353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.966353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:16.966353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.966353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:16.967425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:16.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:16.968327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:16.968327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:16.968327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:16.968327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:17.059363 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:17.060073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:17.060159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:17.060256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:17.060952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:17.061366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:17.061553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:17.062688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:17.062688 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:17.062688 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:17.062688 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:17.062688 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:17.062688 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:17.062688 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:17.062688 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:17.063395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:17.063973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:17.064499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:17.065179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:17.066048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:17.066048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:17.066048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:21.890092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:21.891414 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:21.891682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:21.893171 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:21.921717 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:21.922387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:21.922768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:21.925247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:21.925670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.927636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:21.927636 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.927636 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:21.927636 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.927636 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.927636 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:21.927636 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.927636 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:21.930127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.930688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:21.931480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.931480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:21.931480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:21.932572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.933345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.934128 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:21.935090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.936290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:21.937672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:21.939187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.940295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:21.940295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.940295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.940295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:21.940295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.940295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:21.940295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:21.940295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:21.940295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:21.940295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:21.941165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:21.941165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.941165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:34:21.941165 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.941165 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:21.941165 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.941165 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.941165 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:21.941165 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.941165 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:21.941916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:21.942585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:21.943676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.944446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:21.944949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:21.946492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:21.947723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.947723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:21.947723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.947723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.947723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:21.947723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.947723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:21.959688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:21.960140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:21.960258 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:21.960464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:21.960945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:21.961391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.962424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:34:21.962424 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.962424 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:21.962424 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.962424 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.962424 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:21.962424 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.962424 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:21.963977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:21.964703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:21.964703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:21.964703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.964703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:21.964703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.964703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.964703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:21.964703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.964703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:21.965650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.965650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:21.965650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:21.965650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:21.965650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.965650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:21.965650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:21.965650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:21.965650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:21.965650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.965650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:21.966123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.966684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.966684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:21.966684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.966684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:21.967664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:21.968198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:21.968726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:21.968726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:21.968726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:21.968726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:22.060398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:22.060797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:22.060977 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:22.061284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:22.062435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:22.062904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:22.063115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:22.064193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:22.064193 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:22.064193 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:22.064193 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:22.064193 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:22.064193 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:22.064193 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:22.064193 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:22.065984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:22.066648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:22.067203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:22.068172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:22.069185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:22.069681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:22.069681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:26.891320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:26.892372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:26.892372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:26.892893 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:26.922970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:26.923448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:26.924196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:26.924343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.924674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:26.925514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:26.925514 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.925514 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:26.925514 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.925514 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.925514 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:26.925514 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.925514 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:26.926554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.927314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:26.927880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:26.928497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.929427 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:26.930428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.930989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:26.931388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.931388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:26.932132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.932766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.933380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:26.933380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.933380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:26.938169 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:26.938656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:26.938892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:26.939103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:26.939631 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:26.939917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.940685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:34:26.940685 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.940685 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:26.940685 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.940685 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.940685 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:26.940685 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.940685 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:26.941625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.942805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:26.943165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.943573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.943573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:26.943573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.943573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:26.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:26.944630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:26.945110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.945110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:26.945110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.945110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.945110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:26.945110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.945110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:26.960282 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:26.960774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:26.960902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:26.961057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:26.961389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:26.961619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.962109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:34:26.962109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.962109 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:26.962109 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.962109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.962109 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:26.962109 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.962109 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:26.962915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:26.963614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:26.963614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:26.963614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.963614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:26.963614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.963614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.963614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:26.963614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.963614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:26.964024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.964024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:26.964024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:26.964024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:26.964024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.964024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:26.964024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:26.964024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:26.964024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:26.964024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.964024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:26.964688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.965383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.965383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:26.965383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.965383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:26.965673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:26.966959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:26.967913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:26.967913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:26.967913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:26.967913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:27.060992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:27.061602 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:27.061793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:27.062085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:27.062695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:27.063006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:27.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:27.063997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:27.063997 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:27.063997 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:27.063997 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:27.063997 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:27.063997 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:27.063997 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:27.063997 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:27.065296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:27.065946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:27.066528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:27.066855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:27.067795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:27.068671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:27.068671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:31.891917 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:31.892873 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:31.892873 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:31.894079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:31.924158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:31.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:31.925192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:31.925589 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:31.925767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.926691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:31.926691 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.926691 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:31.926691 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.926691 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.926691 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:31.926691 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.926691 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:31.927777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.928122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:31.928468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.928468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:31.928468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:31.929258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.929751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.930477 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:31.931707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.932921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:31.933866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:31.934904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.935590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:31.935590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.935590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.935590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:31.935590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.935590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:31.938701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:31.939374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:31.939572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:31.939812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:31.940402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:31.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.941722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:34:31.941722 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.941722 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:31.941722 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.941722 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.941722 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:31.941722 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.941722 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:31.943261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:31.944077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:31.944077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:31.944077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.944077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:31.944077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.944077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.944077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:31.944077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.944077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.944882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:31.945850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:31.946680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:31.947270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.948226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:31.960871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:31.961350 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:31.961573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:31.961804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:31.962395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:31.962765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.963567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Traceback (most recent call last):
Apr 21 13:34:31.963567 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.963567 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:31.963567 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.963567 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.963567 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:31.963567 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.963567 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:31.965042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:31.965740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:31.965740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:31.965740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.965740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:31.965740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.965740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.965740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:31.965740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.965740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:31.966807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.966807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:31.966807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:31.966807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:31.966807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.966807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:31.966807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:31.966807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:31.966807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:31.966807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.966807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:31.967297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.967851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.967851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:31.967851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.967851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:31.969315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:31.970038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:31.970563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:31.970563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:31.970563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:31.970563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:32.061534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:32.062017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:32.062143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:32.062341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:32.062820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:32.063177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:32.063392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:32.064074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:32.064074 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:32.064074 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:32.064074 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:32.064074 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:32.064074 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:32.064074 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:32.064074 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:32.065078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:32.065641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:32.066149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:32.066994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:32.067810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:32.067810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:32.067810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:36.892648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:36.893175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:36.893175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:36.893972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:36.920764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=241 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:34:36.925164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:36.926057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:36.926239 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:36.927193 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:36.927862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.929096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:36.929096 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.929096 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:36.929096 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.929096 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.929096 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:36.929096 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.929096 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:36.930371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.930915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:36.931317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.931317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:36.931317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:36.931920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.932720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.933300 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:36.934039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.934382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:36.934707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.934707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:36.935085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.935589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.936057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:36.936057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.936057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:36.939141 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:36.939612 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:36.939768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:36.939930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:36.940404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:36.940579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.941076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:34:36.941076 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.941076 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:36.941076 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.941076 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.941076 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:36.941076 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.941076 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:36.942049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:36.942601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:36.942601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:36.942601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.942601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:36.942601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.942601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.942601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:36.942601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.942601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.943162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:36.943637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.944071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.944071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:36.944071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.944071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:36.945194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:36.945756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:36.946918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.946918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:36.946918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.946918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.946918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:36.946918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.946918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:36.961479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:36.961960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:36.962145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:36.962395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:36.962806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:36.963156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.963923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:34:36.963923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.963923 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:36.963923 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.963923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.963923 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:36.963923 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.963923 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:36.965135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:36.965859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:36.965859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:36.965859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.965859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:36.965859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.965859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.965859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:36.965859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.965859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:36.966302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.966302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:36.966302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:36.966302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:36.966302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.966302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:36.966302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:36.966302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:36.966302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:36.966302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.966302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:36.966822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:36.967448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:36.968088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:36.969841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:36.969841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:36.969841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:36.969841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:36.969841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:36.969841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:36.969841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:36.969841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:36.969841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:36.969841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:36.969841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:37.062127 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:37.062682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:37.062904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:37.063161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:37.063693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:37.064043 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:37.064242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:37.065010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:37.065010 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:37.065010 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:37.065010 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:37.065010 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:37.065010 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:37.065010 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:37.065010 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:37.066516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:37.067092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:37.067619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:37.068192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:37.068998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:37.069856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:37.069856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:41.893445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:41.894344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:41.894344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:41.895609 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:41.921968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:41.922464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.923367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:41.923367 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.923367 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:41.923367 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.923367 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.923367 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:41.923367 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.923367 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:41.924150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.924737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:41.925319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:41.925902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:41.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:41.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:41.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:41.926676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:41.927006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:41.928006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:41.928006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.928931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:41.928931 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.928931 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:41.928931 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.928931 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.928931 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:41.928931 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.928931 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:41.930407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.931051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:41.931732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.931732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:41.932072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.932903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.933855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:41.933855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.933855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:41.939639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:41.940141 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:41.940340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:41.940501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:41.940864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:41.941057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.941592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:34:41.941592 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.941592 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:41.941592 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.941592 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.941592 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:41.941592 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.941592 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:41.942745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:41.943750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:41.944472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.944885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:41.945280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:41.945774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:41.947104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.947104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:41.947104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.947104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.947104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:41.947104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.947104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:41.962092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:41.962578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:41.962766 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:41.962970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:41.963405 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:41.963753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.964480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:34:41.964480 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.964480 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:41.964480 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.964480 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.964480 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:41.964480 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.964480 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:41.965639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:41.967127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:41.967789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:41.968345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:41.968345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:41.968345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:41.968345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:41.968345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.968345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:41.968345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.968345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.968345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:41.968345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.968345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:41.968787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:41.970575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:41.972068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:41.972068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:41.972068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:41.972068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:42.062925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:42.063393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:42.063512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:42.063714 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:42.064198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:42.064668 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:42.064855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:42.065631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:42.065631 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:42.065631 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:42.065631 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:42.065631 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:42.065631 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:42.065631 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:42.065631 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:42.066998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:42.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:42.067851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:42.068492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:42.069044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:42.069496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:42.069496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:46.458096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:46.458862 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:46.458862 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:46.468767 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e3c21a4a-1392-4024-8715-3b16ae581edd None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 21 13:34:46.469057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e3c21a4a-1392-4024-8715-3b16ae581edd None None] Maintenance task: Number of inconsistencies found at create/update: networks=2, security_groups=1 {{(pid=106267) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 21 13:34:46.469186 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e3c21a4a-1392-4024-8715-3b16ae581edd None None] Maintenance task: Fixing resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:34:46.508682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, resource={'id': 'b57c058f-054e-4a2a-980f-6bfda2111e90', 'name': 'private', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['6a3993e6-b5ba-4812-baa3-b25db5880cd9', 'c97deb27-a1d6-4e03-b489-95abde8fd791'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:29:32Z', 'updated_at': '2026-04-21T13:29:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 24380}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:46.508911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:46.509125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3f6c78fa-b671-412d-bb34-5426a40d8bff, external_ids_update=None, columns={'tag': 24380}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:46.509270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.509980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e3c21a4a-1392-4024-8715-3b16ae581edd None None] Traceback (most recent call last):
Apr 21 13:34:46.509980 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.509980 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:46.509980 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.509980 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.509980 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:46.509980 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.509980 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:46.511587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e3c21a4a-1392-4024-8715-3b16ae581edd None None] Maintenance task: Failed to fix resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.511587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:46.511587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:34:46.511587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:34:46.511587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:34:46.511587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:34:46.511587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:34:46.511587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:46.511587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.511587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.511587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.512408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:46.512408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.512408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:46.512408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e3c21a4a-1392-4024-8715-3b16ae581edd None None] Maintenance task: Fixing resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:34:46.539829 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d0607710-681d-444d-be26-dbeca5a07b46, resource={'id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'name': 'public', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-21T13:29:38Z', 'updated_at': '2026-04-21T13:29:48Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:46.540359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d0607710-681d-444d-be26-dbeca5a07b46, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:46.541037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-19f04b40-ed99-482a-8774-a76892a716d1, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:46.541474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e3c21a4a-1392-4024-8715-3b16ae581edd None None] Traceback (most recent call last):
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e3c21a4a-1392-4024-8715-3b16ae581edd None None] Maintenance task: Failed to fix resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:34:46.545619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:46.546142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:46.546599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.546599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:46.546599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.546599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:46.546599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.546599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.546599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:46.546599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.546599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:46.546599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e3c21a4a-1392-4024-8715-3b16ae581edd None None] Maintenance task: Fixing resource d91667b9-d7f5-4dd8-b73d-ec4e87167224 (type: security_groups) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:34:46.612503 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.db.ovn_revision_numbers_db [None req-e3c21a4a-1392-4024-8715-3b16ae581edd None None] Successfully bumped revision number for resource d91667b9-d7f5-4dd8-b73d-ec4e87167224 (type: security_groups) to 9
Apr 21 13:34:46.612702 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e3c21a4a-1392-4024-8715-3b16ae581edd None None] OVN maintenance task check_for_inconsistencies finished in 0.154 seconds
Apr 21 13:34:46.894097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:46.894686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:46.894995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:46.896324 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:46.927070 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:46.927480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:46.927570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:46.928359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:46.928632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.929402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:46.929402 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.929402 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:46.929402 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.929402 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.929402 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:46.929402 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.929402 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.930199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.930817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:46.931360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.931360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:46.931360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:46.931522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.932246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.933070 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.933622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.934156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:46.934788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:46.935463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.935991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:46.935991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.935991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.935991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:46.935991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.935991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:46.940138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:46.940413 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:46.940549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:46.940697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:46.940989 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:46.941163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.941655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:34:46.941655 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.941655 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:46.941655 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.941655 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.941655 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:46.941655 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.941655 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:46.942408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:46.942900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.942900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:46.942900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.942900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:46.942900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.942900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.942900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:46.942900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.942900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.943360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:46.943707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.944060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.944060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:46.944060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.944060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:46.944356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.944758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:46.945235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.945235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:46.945235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.945235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.945235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:46.945235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.945235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:46.962898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:46.963572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:46.963766 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:46.963980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:46.964645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:46.965108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.966616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Traceback (most recent call last):
Apr 21 13:34:46.966616 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.966616 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:46.966616 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.966616 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.966616 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:46.966616 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.966616 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:46.968060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:46.969028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.969028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:46.969028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.969028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:46.969028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.969028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.969028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:46.969028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.969028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:46.969625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.969625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:46.969625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:46.969625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:46.969625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.969625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:46.969625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:46.969625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:46.969625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:46.969625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.969625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:46.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.971291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.971291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:46.971291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.971291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d2a4afb8-d995-4cbd-840a-b3f8abef5972 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:46.971697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:46.972274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:46.972934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:46.972934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:46.972934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:46.972934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:47.063603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:47.064063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:47.064152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:47.064420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:47.065090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:47.065539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:47.065741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:47.066575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:47.066575 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:47.066575 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:47.066575 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:47.066575 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:47.066575 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:47.066575 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:47.066575 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:47.067594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:47.068194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:47.068800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:47.069682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:47.070594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:47.070594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:47.070594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:51.894790 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:51.895506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:51.895782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:51.897110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:51.928486 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:51.928943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:51.929064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:51.931170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:51.932579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.935898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:51.935898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.935898 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:51.935898 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.935898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.935898 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:51.935898 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.935898 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:51.937006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.937629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:51.938545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:51.939284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.940079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:51.940079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.940079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.940079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:51.940079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.940079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:51.941945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:51.943207 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:51.943525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:51.944109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:51.944693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:51.945020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.945753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:51.945968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.946417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:51.946417 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.946417 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:51.946417 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.946417 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.946417 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:51.946417 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.946417 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:51.947157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.947633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:51.948084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.948084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:51.948274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.951742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:51.952289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.952881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.953393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:51.954240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.954645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.954645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:51.954645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.954645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:51.955156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:51.955630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:51.956131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.956131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:51.956131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.956131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.956131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:51.956131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.956131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:51.963406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:51.964057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:51.964293 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:51.964527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:51.965186 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:51.965603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:51.966440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.966440 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:51.966440 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.966440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.966440 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:51.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:51.968199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:51.968701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:51.968701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.968701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:51.968701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.968701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.968701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:51.968701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.968701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:51.969673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.970054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.970483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:51.970483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.970483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:51.971543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:51.972049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:51.972581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:51.972581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:51.972581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:51.972581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:52.064390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:52.065255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:52.065255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:52.065255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:52.065660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:52.066048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:52.066239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:52.068898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:52.069489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:52.069836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:52.070279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:52.070862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:52.070862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:52.070862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:52.070862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:52.070862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:52.070862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:52.070862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:52.070862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:52.070862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:56.895469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:56.896188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:56.896188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:56.896985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:56.926344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:56.926766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.927774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:34:56.927774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.927774 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:56.927774 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.927774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.927774 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:56.927774 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.927774 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:56.928572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.928939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:56.929433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.929433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:56.929538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.930040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.930667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:56.930667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.930667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:56.930667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:56.930667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:56.930667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:56.930889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:56.931246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.932163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:34:56.932163 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.932163 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:56.932163 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.932163 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.932163 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:56.932163 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.932163 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:56.933794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.934183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:56.934643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.934643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:56.935609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.936239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.936820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:56.936820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.936820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:56.943242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:56.943741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:56.943946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:56.944161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:34:56.944823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:56.945277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.946185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:34:56.946185 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.946185 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:56.946185 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.946185 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.946185 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:56.946185 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.946185 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:56.947511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:56.948389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:56.948389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:56.948389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.948389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:56.948389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.948389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.948389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:56.948389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.948389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.949038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:56.949691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.950378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.950378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:56.950378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.950378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:56.950803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:56.951669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:56.952272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.952272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:56.952272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.952272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.952272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:56.952272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.952272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:56.964033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:56.964601 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:56.964741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:56.964906 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:56.965436 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:56.965708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.966299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:56.966299 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.966299 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:56.966299 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.966299 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.966299 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:56.966299 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.966299 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:34:56.967633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.968591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:56.969044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:56.969451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:56.969451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:56.969451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.969451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:56.969451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.969451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.969451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:56.969451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.969451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:56.969738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:56.970256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:56.970780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:56.970780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:56.970780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:56.970780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:34:57.065163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:34:57.065630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:34:57.065725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:34:57.065898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:34:57.066479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:57.066783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:34:57.066926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:57.067588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:34:57.067588 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:57.067588 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:34:57.067588 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:34:57.067588 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:57.067588 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:34:57.067588 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:57.067588 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:57.068668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:34:57.069076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:34:57.069703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:34:57.070472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:34:57.071302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:34:57.071302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:34:57.071302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:01.896870 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:01.897818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:01.897818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:01.898552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:01.930801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:01.931809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:01.932732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:01.932985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.933451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:01.934524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:01.934524 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.934524 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:01.934524 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.934524 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.934524 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:01.934524 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.934524 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:01.936042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.936605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:01.937052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.937052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:01.937052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:01.938729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.939245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.939842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:01.939842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.939842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:01.939842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.939976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:01.939976 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.939976 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:01.939976 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.939976 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.939976 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:01.939976 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.939976 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:01.941556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.942105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:01.942597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.942597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:01.943206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.943888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.944489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:01.944489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.944489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:01.944489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:01.944489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:01.944685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:01.944890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:01.945373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:01.945559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.946167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:01.946167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.946167 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:01.946167 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.946167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.946167 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:01.946167 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.946167 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:01.947714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:01.948267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:01.948267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:01.948267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.948267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:01.948267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.948267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.948267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:01.948267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.948267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.949448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:01.949858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.950295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.950295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:01.950295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.950295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:01.951224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:01.951862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:01.952393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.952393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:01.952393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.952393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.952393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:01.952393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.952393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:01.964626 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:01.964954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:01.965061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:01.965227 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:01.965836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:01.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.966961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:01.966961 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.966961 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:01.966961 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.966961 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.966961 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:01.966961 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.966961 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:01.967984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.968829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:01.969815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:01.970696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:01.971614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:01.972779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:02.065973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:02.066580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:02.066725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:02.067030 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:02.067914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:02.068534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:02.068740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:02.069555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:02.069555 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:02.069555 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:02.069555 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:02.069555 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:02.069555 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:02.069555 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:02.069555 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:02.070918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:02.071564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:02.072158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:02.072478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:02.073247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:02.074122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:02.074122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:06.897373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:06.898031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:06.898031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:06.899617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:06.932823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:06.933955 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:06.934381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:06.936475 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:06.937125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.938472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:06.938472 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.938472 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:06.938472 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.938472 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.938472 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:06.938472 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.938472 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:06.940309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.940911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:06.941441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.941441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:06.941441 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:06.942138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.943934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:06.945464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:06.946125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:06.947432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:06.948048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:06.948633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:06.949049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:06.949049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:06.949049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:06.949049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:06.949049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.949580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:06.949580 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.949580 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:06.949580 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.949580 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.949580 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:06.949580 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.949580 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:06.950676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:06.951134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:06.951134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:06.951134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.951134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:06.951134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.951134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.951134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:06.951134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.951134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.951648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:06.951985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.952392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.952392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:06.952392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.952392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:06.952953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:06.953410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:06.953893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.953893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:06.953893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.953893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.953893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:06.953893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.953893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:06.965156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:06.965499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:06.965592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:06.965704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:06.966523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:06.967114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:06.967861 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.967861 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:06.967861 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.967861 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.967861 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:06.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:06.969654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:06.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:06.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:06.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:06.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.970367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:06.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.971931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.972494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:06.972494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.972494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:06.973382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:06.974074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:06.974937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:06.974937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:06.974937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:06.974937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:07.075420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:07.075420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:07.075420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:07.075420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:07.075420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:07.075420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:07.075748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:07.076317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:07.076769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:07.077306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:07.078002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:07.078002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:07.078002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:07.078002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:07.078002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:07.078002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:07.078002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:07.078002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:07.078002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:11.898747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:11.899395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:11.899491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:11.900670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:11.933913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:11.934584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:11.934767 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:11.935723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:11.937058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.937348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:11.937348 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.937348 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:11.937348 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.937348 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.937348 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:11.937348 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.937348 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:11.938664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.939159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:11.939818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.939818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:11.940846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.942111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.942870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:11.942870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.942870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:11.942870 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:11.942870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.944447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:11.944447 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.944447 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:11.944447 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.944447 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.944447 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:11.944447 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.944447 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:11.946139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.946637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:11.947149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.947149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:11.950581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.951124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.951871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:11.951871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.951871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:11.951871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:11.951871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:11.951871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:11.951871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:11.953019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:11.953019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.953439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:11.953439 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.953439 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:11.953439 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.953439 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.953439 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:11.953439 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.953439 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:11.954668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:11.955204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:11.955204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:11.955204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.955204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:11.955204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.955204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.955204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:11.955204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.955204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.955950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:11.956420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.956855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.956855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:11.956855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.956855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:11.957649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:11.958101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:11.958684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.958684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:11.958684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.958684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.958684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:11.958684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.958684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:11.965671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:11.966194 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:11.966444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:11.966699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:11.967344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:11.967683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.968357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:35:11.968357 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.968357 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:11.968357 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.968357 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.968357 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:11.968357 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.968357 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:11.969570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:11.970286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:11.970286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:11.970286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.970286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:11.970286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.970286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.970286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:11.970286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.970286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:11.971020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.971020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:11.971020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:11.971020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:11.971020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.971020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:11.971020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:11.971020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:11.971020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:11.971020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.971020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:11.971589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.972307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.972307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:11.972307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.972307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:11.972963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:11.973866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:11.974812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:11.974812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:11.974812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:11.974812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:12.067132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:12.067845 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:12.067957 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:12.068183 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:12.068887 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:12.069426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:12.069664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:12.070606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:12.070606 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:12.070606 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:12.070606 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:12.070606 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:12.070606 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:12.070606 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:12.070606 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:12.071977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:12.072571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:12.073096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:12.073475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:12.074174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:12.074971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:12.074971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:16.899457 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:16.900512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:16.900512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:16.901488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:16.933713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:16.934072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.935503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:16.935945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:16.935945 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.935945 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:16.935945 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.935945 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.935945 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:16.935945 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.935945 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:16.936975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.937570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:16.938175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:16.938833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:16.939909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.940837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:16.940837 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.940837 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:16.940837 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.940837 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.940837 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:16.940837 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.940837 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:16.942807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.943154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:16.943540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.943540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:16.944715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.945166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.945736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:16.945736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.945736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:16.951533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:16.952250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:16.952409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:16.952637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:16.953024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:16.953201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.953849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:16.953849 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.953849 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:16.953849 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.953849 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.953849 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:16.953849 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.953849 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:16.955254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:16.956094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:16.956094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:16.956094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.956094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:16.956094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.956094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.956094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:16.956094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.956094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.956580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:16.957153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.958076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.958076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:16.958076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.958076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:16.958405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:16.959353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:16.960177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.960177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:16.960177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.960177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.960177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:16.960177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.960177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:16.966331 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:16.966876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:16.967000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:16.967117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:16.967600 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:16.967921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.969358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:35:16.969358 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.969358 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:16.969358 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.969358 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.969358 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:16.969358 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.969358 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:16.970179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:16.970898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:16.970898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:16.970898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.970898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:16.970898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.970898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.970898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:16.970898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.970898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:16.971320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.971320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:16.971320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:16.971320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:16.971320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.971320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:16.971320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:16.971320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:16.971320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:16.971320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.971320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:16.971980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:16.972803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:16.973686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:16.974707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:16.974707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:16.974707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:16.974707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:16.974707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:16.974707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:16.974707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:16.974707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:16.974707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:16.974707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:16.974707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:17.067984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:17.068532 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:17.068671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:17.068892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:17.069674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:17.070168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:17.070500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:17.071405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:17.071405 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:17.071405 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:17.071405 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:17.071405 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:17.071405 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:17.071405 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:17.071405 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:17.073012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:17.073546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:17.073887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:17.075038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:17.075684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:17.076164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:17.076164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:21.900052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:21.901321 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:21.901464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:21.903025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:21.936016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:21.936145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.936769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:21.936769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.936769 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:21.936769 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.936769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.936769 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:21.936769 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.936769 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:21.937557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.937881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:21.938312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.938312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:21.939155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.939695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.940353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:21.940353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.940353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:21.940353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:21.940353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:21.940353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:21.941144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:21.941363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.942011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:21.942011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.942011 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:21.942011 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.942011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.942011 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:21.942011 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.942011 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:21.944132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.944624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:21.945037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.945037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:21.946191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.946731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.947233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:21.947233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.947233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:21.951980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:21.953644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:21.953759 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:21.953855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:21.954390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:21.954600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.955251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:21.955251 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.955251 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:21.955251 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.955251 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.955251 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:21.955251 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.955251 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:21.956032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:21.956956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:21.956956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:21.956956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.956956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:21.956956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.956956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.956956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:21.956956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.956956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.958580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:21.959169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.959854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.959854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:21.959854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.959854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:21.962102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:21.962738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:21.963239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.963239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:21.963239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.963239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.963239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:21.963239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.963239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:21.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:21.967576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:21.967576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:21.967576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:21.967861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:21.968095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.968998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Traceback (most recent call last):
Apr 21 13:35:21.968998 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.968998 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:21.968998 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.968998 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.968998 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:21.968998 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.968998 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:21.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:21.970282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:21.970282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:21.970282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.970282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:21.970282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.970282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.970282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:21.970282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.970282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:21.971449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.971449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:21.971449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:21.971449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:21.971449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.971449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:21.971449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:21.971449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:21.971449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:21.971449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.971449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:21.971768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.972104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.972104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:21.972104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.972104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:21.973495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:21.974009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:21.974485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:21.974485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:21.974485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:21.974485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:22.068597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:22.068957 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:22.069063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:22.069201 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:22.069944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:22.070354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:22.070536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:22.071316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:22.071316 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:22.071316 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:22.071316 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:22.071316 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:22.071316 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:22.071316 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:22.071316 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:22.072317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:22.072954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:22.073529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:22.074256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:22.075044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:22.075044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:22.075044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:26.900885 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:26.901820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:26.902432 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:26.903239 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:26.936267 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:26.936624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.937466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:26.937466 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.937466 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:26.937466 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.937466 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.937466 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:26.937466 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.937466 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:26.938148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.938914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:26.939723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:26.940901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:26.942064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:26.942679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:26.943348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.943829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:26.943829 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.943829 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:26.943829 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.943829 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.943829 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:26.943829 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.943829 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:26.944926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.945430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:26.945917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.945917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:26.946061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.946757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.947505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:26.947505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.947505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:26.953648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:26.954510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:26.954662 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:26.954845 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:26.955102 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:26.955269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.955801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:26.955801 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.955801 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:26.955801 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.955801 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.955801 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:26.955801 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.955801 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:26.957218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:26.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:26.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:26.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:26.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:26.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.958584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:26.958984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.959393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.959393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:26.959393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.959393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:26.960147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:26.960633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:26.961196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.961196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:26.961196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.961196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.961196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:26.961196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.961196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:26.967408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:26.967932 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:26.968155 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:26.968820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:26.969447 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:26.970144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.971199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Traceback (most recent call last):
Apr 21 13:35:26.971199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.971199 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:26.971199 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.971199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.971199 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:26.971199 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.971199 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:26.973933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:26.974748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:26.974748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:26.974748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.974748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:26.974748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.974748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.974748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:26.974748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.974748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:26.975163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.975163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:26.975163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:26.975163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:26.975163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.975163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:26.975163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:26.975163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:26.975163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:26.975163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.975163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:26.975730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:26.976497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:26.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:26.978610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:26.978610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:26.978610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:26.978610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:26.978610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:26.978610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:26.978610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:26.978610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:26.978610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:26.978610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:26.978610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:27.069243 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:27.070327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:27.070327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:27.070327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:27.071179 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:27.071578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:27.071760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:27.073252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:27.073252 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:27.073252 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:27.073252 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:27.073252 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:27.073252 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:27.073252 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:27.073252 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:27.074339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:27.075062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:27.075825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:27.076619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:27.078879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:27.078879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:27.078879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:27.292706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=ece32a62-45f1-4d59-9397-45ceee3459cd, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-d0607710-681d-444d-be26-dbeca5a07b46', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-06899d4c-a436-45e4-bed5-95dabe888f51', 'neutron:subnet_ids': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3 ca2ae363-6d7d-4b39-b330-44fb15b7cd49'}, gateway_chassis=[], ha_chassis_group=243633f2-900a-4202-9f17-3f296606b229, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:fb:71:35', name='lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', networks=['172.24.5.49/24', '2001:db8::162/64'], options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=106267) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 21 13:35:29.366149 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=5cafa7c3-a999-4ffc-86fd-58793024d1fb, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-06899d4c-a436-45e4-bed5-95dabe888f51', 'neutron:subnet_ids': '620e9559-2c15-4370-904f-f4677bc69542'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:b9:b3:62', name='lrp-f3ac525a-3cf4-418a-a396-dbcbf5dd0f02', networks=['192.168.24.1/24'], options={}, peer=[], status={}) old= {{(pid=106267) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 21 13:35:29.401430 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377 synchronized; highest priority chassis 27f467c0-a74a-4bc5-9cac-d4ea2716e85a
Apr 21 13:35:29.402118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': 'b364ad09-f70d-45d8-b9dd-8420f5f1c377', 'neutron:router_id': '06899d4c-a436-45e4-bed5-95dabe888f51'}}) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:29.402544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, chassis=27f467c0-a74a-4bc5-9cac-d4ea2716e85a, priority=1, columns={}) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:29.402806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:29.403408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:29.403408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:29.403408 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:29.403408 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:29.403408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:29.403408 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:29.403408 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:29.403408 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2411, in link_network_ha_chassis_group
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     next(self.gen)
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     next(self.gen)
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 21 13:35:29.404190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 21 13:35:29.404901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:29.404901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     raise result.ex
Apr 21 13:35:29.404901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:29.404901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 21 13:35:29.404901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 21 13:35:29.404901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:29.404901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 21 13:35:29.404901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:29.404901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.event 
Apr 21 13:35:31.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:31.902454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:31.903494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:31.904109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:31.935805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:31.936137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.936851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:31.936851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.936851 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:31.936851 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.936851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.936851 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:31.936851 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.936851 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:31.937534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.937866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:31.938225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.938225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:31.938349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.939068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.939579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:31.939579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.939579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:31.940414 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:31.941054 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:31.941289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:31.942285 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:31.942669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:31.942977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.943946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:31.943946 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.943946 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:31.943946 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.943946 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.943946 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:31.943946 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.943946 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:31.945586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.946077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:31.946611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.946611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:31.947736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.948612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.949326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:31.949326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.949326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:31.954459 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:31.954696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:31.954800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:31.954923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:31.955388 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:31.955693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.956340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:31.956340 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.956340 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:31.956340 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.956340 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.956340 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:31.956340 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.956340 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:31.957065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:31.957706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:31.957706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:31.957706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.957706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:31.957706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.957706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.957706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:31.957706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.957706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.958062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:31.958553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:31.959137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:31.959687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.960476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:31.968740 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:31.969076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:31.969225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:31.969759 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:31.970466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:31.970466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Traceback (most recent call last):
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.973546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:31.975107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.975107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:31.975107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:31.975107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:31.975107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.975107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:31.975107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:31.975107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:31.975107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:31.975107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.975107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:31.975654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:31.977388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:31.978032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:31.979203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:31.979203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:31.979203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:31.979203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:31.979203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:31.979203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:31.979203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:31.979203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:31.979203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:31.979203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:31.979203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:32.070289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:32.070796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:32.070950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:32.071195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:32.071820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:32.072205 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:32.072462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:32.074369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:32.074369 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:32.074369 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:32.074369 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:32.074369 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:32.074369 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:32.074369 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:32.074369 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:32.075649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:32.076405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:32.077260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:32.077954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:32.078980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:32.080330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:32.080330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:36.902510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:36.903255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:36.903466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:36.905224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:36.929666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=317,neutron:UPDATE=1 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:35:36.941440 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:36.942355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:36.942466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:36.944463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:36.944774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:36.944997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.946878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:36.946878 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.946878 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:36.946878 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.946878 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.946878 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:36.946878 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.946878 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:36.949198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.949661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:36.950018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.950018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:36.950018 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:36.953044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.953596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.954167 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:36.955415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.955923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:36.956354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.956354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:36.956354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:36.956924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.957484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.958175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:36.958175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.958175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:36.958175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:36.958175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:36.958175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:36.958175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:36.958538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.959201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:36.959201 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.959201 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:36.959201 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.959201 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.959201 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:36.959201 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.959201 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:36.960359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:36.961009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:36.961009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:36.961009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.961009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:36.961009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.961009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.961009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:36.961009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.961009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.961581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:36.962130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:36.962743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:36.963304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.963937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:36.969132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:36.970411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:36.970411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:36.970411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:36.973490 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:36.973834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.974665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:35:36.974665 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.974665 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:36.974665 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.974665 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.974665 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:36.974665 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.974665 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:36.976131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:36.976917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:36.976917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:36.976917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.976917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:36.976917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.976917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.976917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:36.976917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.976917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:36.978139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.978139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:36.978139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:36.978139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:36.978139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.978139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:36.978139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:36.978139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:36.978139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:36.978139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.978139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:36.978912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:36.980026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:36.980980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:36.982130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:36.982130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:36.982130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:36.982130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:36.982130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:36.982130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:36.982130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:36.982130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:36.982130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:36.982130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:36.982130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:37.070971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:37.071264 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:37.071409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:37.071620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:37.072383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:37.072770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:37.073000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:37.075920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:37.075920 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:37.075920 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:37.075920 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:37.075920 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:37.075920 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:37.075920 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:37.075920 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:37.076912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:37.077479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:37.077869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:37.078000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:37.078457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:37.079062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:37.079062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:41.903405 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:41.904819 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:41.905043 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:41.906386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:41.944307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:41.948076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:41.949880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:41.951314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:41.951314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.951486 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:41.951701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:41.951866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.952729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:41.952729 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.952729 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:41.952729 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.952729 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.952729 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:41.952729 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.952729 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:41.954037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.954306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:41.954563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.954563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:41.954563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:41.954563 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.954563 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:41.954563 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.954563 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.954563 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:41.954563 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.954563 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:41.955543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.955876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.956234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:41.956234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.956234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:41.956842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.957066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:41.957754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.957754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:41.958341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.958657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.959092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:41.959092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.959092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:41.959092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:41.959092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:41.959092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:41.959092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:41.959453 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:41.959641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.961548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:41.961548 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.961548 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:41.961548 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.961548 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.961548 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:41.961548 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.961548 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:41.962131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:41.962460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:41.962460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:41.962460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.962460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:41.962460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.962460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.962460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:41.962460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.962460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.962740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:41.963049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.963356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.963356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:41.963356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.963356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:41.963467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:41.963750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:41.964064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.964064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:41.964064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.964064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.964064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:41.964064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.964064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:41.969916 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:41.970315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:41.970423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:41.970615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:41.971017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:41.971387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.972298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Traceback (most recent call last):
Apr 21 13:35:41.972298 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.972298 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:41.972298 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.972298 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.972298 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:41.972298 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.972298 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:41.973239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:41.973600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:41.973600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:41.973600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.973600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:41.973600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.973600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.973600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:41.973600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.973600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:41.974142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.974142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:41.974142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:41.974142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:41.974142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.974142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:41.974142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:41.974142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:41.974142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:41.974142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.974142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:41.974399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.974669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.974669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:41.974669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.974669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:41.975518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:41.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:41.976370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:41.976370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:41.976370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:41.976370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:42.071637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:42.071768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:42.071768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:42.071847 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:42.072280 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:42.072484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:42.072594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:42.073055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:42.073055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:42.073055 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:42.073055 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:42.073055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:42.073055 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:42.073055 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:42.073055 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:42.074015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:42.074304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:42.074557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:42.074684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:42.074991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:42.075336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:42.075336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:46.904763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:46.905389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:46.905389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:46.905943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:46.938511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:46.938929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.939716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:46.939716 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.939716 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:46.939716 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.939716 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.939716 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:46.939716 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.939716 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:46.940678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.941549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:46.942424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:46.943038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.943843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:46.943843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.943843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.943843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:46.943843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.943843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:46.948106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:46.949123 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:46.949357 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:46.950592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:46.950998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:46.951335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.952965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:46.952965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.952965 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:46.952965 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.952965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.952965 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:46.952965 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.952965 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:46.954811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.955669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:46.956409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.956409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:46.956545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.957407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.959462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:46.959462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.959462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:46.959462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:46.959855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:46.960113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:46.960320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:46.961190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:46.961534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.962432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:46.962432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.962432 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:46.962432 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.962432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.962432 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:46.962432 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.962432 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:46.963461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:46.963859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:46.963859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:46.963859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.963859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:46.963859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.963859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.963859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:46.963859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.963859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.964337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:46.964881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.965268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.965268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:46.965268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.965268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:46.965669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:46.966098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:46.967172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.967172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:46.967172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.967172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.967172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:46.967172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.967172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:46.970504 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:46.971748 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:46.971888 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:46.972038 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:46.973105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:46.973649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:35:46.974843 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.974843 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:46.974843 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.974843 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.974843 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:46.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:46.976096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:46.976636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:46.976636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:46.976636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.976636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:46.976636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.976636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.976636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:46.976636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.976636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:46.978302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.978302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:46.978302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:46.978302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:46.978302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.978302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:46.978302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:46.978302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:46.978302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:46.978302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.978302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:46.978723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.979290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.979290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:46.979290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.979290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:46.980187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:46.980719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:46.981760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:46.981760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:46.981760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:46.981760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:47.072229 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:47.072821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:47.072968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:47.073196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:47.074334 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:47.074773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:47.075013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:47.076041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:47.076041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:47.076041 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:47.076041 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:47.076041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:47.076041 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:47.076041 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:47.076041 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:47.077701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:47.078402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:47.079052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:47.079370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:47.080223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:47.083313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:47.083313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:51.906254 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:51.907044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:51.907340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:51.908562 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:51.944125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:51.944552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.945364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:51.945364 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.945364 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:51.945364 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.945364 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.945364 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:51.945364 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.945364 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:51.946897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.947438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:51.947967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:51.948586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.949329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:51.949329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.949329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.949329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:51.949329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.949329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:51.949329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:51.949684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:51.950304 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:51.951251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:51.951251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:51.951473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.952193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:51.952193 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.952193 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:51.952193 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.952193 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.952193 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:51.952193 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.952193 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:51.953872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.954555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:51.955232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.955232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:51.955930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.957016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.957805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:51.957805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.957805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:51.959789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:51.960419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:51.960540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:51.960685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:51.961446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:51.961845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.962661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:51.962661 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.962661 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:51.962661 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.962661 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.962661 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:51.962661 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.962661 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:51.963550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:51.963917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:51.963917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:51.963917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.963917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:51.963917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.963917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.963917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:51.963917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.963917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.964417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:51.965600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:51.966397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:51.966752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.967197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:51.971747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:51.972475 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:51.972739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:51.972976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:51.973880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:51.974301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.975114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Traceback (most recent call last):
Apr 21 13:35:51.975114 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.975114 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:51.975114 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.975114 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.975114 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:51.975114 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.975114 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:51.976193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:51.976751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:51.976751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:51.976751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.976751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:51.976751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.976751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.976751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:51.976751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.976751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:51.977508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.977508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:51.977508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:51.977508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:51.977508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.977508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:51.977508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:51.977508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:51.977508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:51.977508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.977508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:51.977979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.978446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.978446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:51.978446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.978446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:51.979565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:51.980046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:51.980545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:51.980545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:51.980545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:51.980545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:52.073110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:52.073561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:52.073711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:52.073953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:52.074885 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:52.075336 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:52.075579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:52.076489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:52.076489 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:52.076489 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:52.076489 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:52.076489 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:52.076489 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:52.076489 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:52.076489 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:52.078069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:52.078674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:52.079374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:52.079924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:52.080739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:52.081790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:52.081790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:56.907030 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:56.907942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:56.907942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:56.909254 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:56.944084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:56.944435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.945093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:35:56.945093 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.945093 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:56.945093 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.945093 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.945093 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:56.945093 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.945093 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:56.946300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.946820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:56.947420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:56.948051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.948832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:56.948832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.948832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.948832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:56.948832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.948832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:56.950392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:56.950644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:56.950782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:56.951349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:56.951618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:56.951784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.954432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:35:56.954432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.954432 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:56.954432 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.954432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.954432 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:56.954432 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.954432 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:56.955225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.955918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:56.956679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:56.957519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.958548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:56.958548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.958548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.958548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:56.958548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.958548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:56.960407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:56.960857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:56.960986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:56.961124 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:35:56.961694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:56.962114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.962823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:35:56.962823 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.962823 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:56.962823 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.962823 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.962823 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:56.962823 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.962823 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:56.963642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:56.964474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:56.965141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.965812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:56.966486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:56.966915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:56.968159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.968159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:56.968159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.968159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.968159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:56.968159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.968159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:56.972176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:56.972699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:56.972872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:56.973098 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:56.973631 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:56.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.975059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:35:56.975059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.975059 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:56.975059 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.975059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.975059 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:56.975059 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.975059 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:35:56.976447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:35:56.977367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:56.977367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:35:56.977367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.977367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:35:56.977367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.977367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.977367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:35:56.977367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.977367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:35:56.977985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.977985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:56.977985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:56.977985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:56.977985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.977985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:56.977985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:56.977985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:56.977985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:56.977985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.977985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:56.978626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.979368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.979368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:56.979368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.979368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:56.979743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:56.980832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:56.981900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:56.981900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:56.981900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:56.981900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:35:57.073840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:35:57.074543 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:35:57.074708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:35:57.074921 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:35:57.075531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:57.075838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:35:57.075993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:57.076735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:35:57.076735 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:57.076735 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:35:57.076735 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:35:57.076735 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:57.076735 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:35:57.076735 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:57.076735 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:57.080628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:35:57.081405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:35:57.081956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:35:57.082775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:35:57.083708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:35:57.083708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:35:57.083708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:01.907898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:01.908639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:01.908639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:01.909435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:01.944036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:01.944634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.945538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:01.945538 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.945538 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:01.945538 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.945538 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.945538 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:01.945538 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.945538 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:01.946350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.946934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:01.947590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:01.948248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.949075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:01.949075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.949075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.949075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:01.949075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.949075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:01.950525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:01.950973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:01.951093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:01.952264 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:01.952443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:01.952702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.953537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:36:01.953537 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.953537 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:01.953537 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.953537 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.953537 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:01.953537 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.953537 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:01.954663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.955046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:01.955406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.955406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:01.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.956390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.956909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:01.956909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.956909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:01.960907 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:01.961680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:01.961835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:01.962025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:01.962531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:01.962835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.963581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:36:01.963581 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.963581 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:01.963581 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.963581 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.963581 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:01.963581 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.963581 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:01.964560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:01.965202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:01.965202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:01.965202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.965202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:01.965202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.965202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.965202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:01.965202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.965202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.965771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:01.966241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.966802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.966802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:01.966802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.966802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:01.967206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:01.967758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:01.968251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.968251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:01.968251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.968251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.968251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:01.968251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.968251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:01.972644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:01.973301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:01.973500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:01.973727 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:01.974621 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:01.975031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.975994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:36:01.975994 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.975994 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:01.975994 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.975994 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.975994 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:01.975994 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.975994 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:01.977398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:01.978116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:01.978116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:01.978116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.978116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:01.978116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.978116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.978116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:01.978116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.978116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:01.978880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.978880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:01.978880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:01.978880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:01.978880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.978880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:01.978880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:01.978880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:01.978880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:01.978880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.978880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:01.979494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.980171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.980171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:01.980171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.980171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:01.981010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:01.982181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:01.983138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:01.983138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:01.983138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:01.983138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:02.074665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:02.075118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:02.075253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:02.075547 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:02.076232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:02.076748 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:02.076975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:02.077950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:02.077950 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:02.077950 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:02.077950 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:02.077950 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:02.077950 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:02.077950 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:02.077950 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:02.079326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:02.080093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:02.080692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:02.080927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:02.081435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:02.081921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:02.081921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:06.908376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:06.909685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:06.909685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:06.910653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:06.952315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:06.952688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:06.952949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:06.952949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:06.952949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:06.952949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.952949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:06.952949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.952949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.952949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:06.952949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.952949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:06.952949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:06.953707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.954237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.954707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:06.954707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.954707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:06.954707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:06.954707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:06.954813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:06.956921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:06.957393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:06.957770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:06.958056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.958418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:06.961747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:06.962383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:06.962580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:06.962782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:06.963829 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:06.964200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.965101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:36:06.965101 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.965101 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:06.965101 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.965101 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.965101 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:06.965101 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.965101 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:06.966724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:06.967257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:06.967257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:06.967257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.967257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:06.967257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.967257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.967257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:06.967257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.967257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.969055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:06.969545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.969967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.969967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:06.969967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.969967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:06.971611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:06.972168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:06.972715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.972715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:06.972715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.972715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.972715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:06.972715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.972715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:06.973200 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:06.974149 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:06.974496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:06.974827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:06.975552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:06.975922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.977011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Traceback (most recent call last):
Apr 21 13:36:06.977011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.977011 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:06.977011 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.977011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.977011 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:06.977011 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.977011 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:06.978787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:06.979308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:06.979308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:06.979308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.979308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:06.979308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.979308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.979308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:06.979308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.979308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:06.979768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.979768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:06.979768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:06.979768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:06.979768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.979768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:06.979768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:06.979768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:06.979768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:06.979768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.979768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:06.980149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.980678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.980678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:06.980678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.980678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:06.981398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:06.981997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:06.982865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:06.982865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:06.982865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:06.982865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:07.075502 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:07.075717 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:07.075812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:07.075980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:07.076572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:07.077008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:07.077304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:07.078204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:07.078204 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:07.078204 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:07.078204 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:07.078204 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:07.078204 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:07.078204 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:07.078204 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:07.079795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:07.080613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:07.081169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:07.081590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:07.082394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:07.083245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:07.083245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:11.909190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:11.910080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:11.910080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:11.910993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:11.951428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:11.951651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.952676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:11.952676 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.952676 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:11.952676 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.952676 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.952676 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:11.952676 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.952676 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:11.953709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.954699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:11.955534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:11.956518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:11.957412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:11.957942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:11.957942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.957942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:36:11.957942 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.957942 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:11.957942 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.957942 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.957942 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:11.957942 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.957942 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:11.959503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.960240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:11.960860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.960860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:11.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.962098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.962609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:11.962609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.962609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:11.962609 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:11.962800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:11.963125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:11.963434 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:11.963865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:11.964106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.964959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:36:11.964959 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.964959 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:11.964959 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.964959 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.964959 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:11.964959 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.964959 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:11.965690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:11.966275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:11.966275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:11.966275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.966275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:11.966275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.966275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.966275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:11.966275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.966275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.966559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:11.967095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:11.967580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:11.968085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.968660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:11.973909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:11.974698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:11.974930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:11.975167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:11.975618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:11.975881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.976821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:36:11.976821 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.976821 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:11.976821 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.976821 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.976821 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:11.976821 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.976821 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:11.978492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:11.979523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:11.979523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:11.979523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.979523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:11.979523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.979523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.979523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:11.979523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.979523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:11.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:11.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:11.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:11.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:11.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:11.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:11.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:11.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:11.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:11.981908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:11.982825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:11.984009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:11.984009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:11.984009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:11.984009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:11.984009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:11.984009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:11.984009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:11.984009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:11.984009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:11.984009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:11.984009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:12.076200 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:12.076681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:12.076831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:12.076956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:12.077554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:12.077960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:12.078308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:12.079152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:12.079152 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:12.079152 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:12.079152 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:12.079152 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:12.079152 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:12.079152 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:12.079152 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:12.080253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:12.080910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:12.081805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:12.082583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:12.083476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:12.083476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:12.083476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:16.909890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:16.910924 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:16.911079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:16.912061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:16.953265 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:16.953777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.954870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:16.954870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.954870 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:16.954870 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.954870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.954870 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:16.954870 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.954870 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:16.955885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.956376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:16.956844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.956844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:16.956844 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:16.957450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.958699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.959850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:16.959850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.959850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:16.959850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:16.959850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:16.959850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:16.959850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:16.959850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:16.960586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:16.961971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.962498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:16.963004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:16.963678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:16.964163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:16.964163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:16.964786 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:16.965115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.965981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:36:16.965981 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.965981 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:16.965981 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.965981 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.965981 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:16.965981 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.965981 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:16.967145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:16.968408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:16.968904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.969245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:16.969670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:16.970079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:16.970533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.970533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:16.970533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.970533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.970533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:16.970533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.970533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:16.974451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:16.975132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:16.975411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:16.975630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:16.976083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:16.976429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.977167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Traceback (most recent call last):
Apr 21 13:36:16.977167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.977167 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:16.977167 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.977167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.977167 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:16.977167 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.977167 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:16.979393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:16.979802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:16.979802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:16.979802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.979802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:16.979802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.979802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.979802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:16.979802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.979802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:16.981101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.981101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:16.981101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:16.981101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:16.981101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.981101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:16.981101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:16.981101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:16.981101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:16.981101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.981101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:16.981431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.981782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.981782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:16.981782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.981782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:16.983248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:16.983724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:16.984197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:16.984197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:16.984197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:16.984197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:17.077109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:17.077655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:17.077829 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:17.078172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:17.078845 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:17.079316 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:17.079598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:17.080575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:17.080575 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:17.080575 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:17.080575 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:17.080575 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:17.080575 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:17.080575 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:17.080575 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:17.082515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:17.082948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:17.083287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:17.084844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:17.085404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:17.085935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:17.085935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:21.910916 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:21.911736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:21.911736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:21.912313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:21.942429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:21.942917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.943810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:21.943810 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.943810 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:21.943810 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.943810 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.943810 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:21.943810 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.943810 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:21.944809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.945413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:21.945961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:21.946739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.947548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:21.947548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.947548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.947548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:21.947548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.947548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:21.957489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:21.958079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:21.958453 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:21.959346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:21.959665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:21.960090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.960686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:36:21.960686 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.960686 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:21.960686 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.960686 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.960686 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:21.960686 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.960686 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:21.962015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.962683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:21.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:21.963667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.964493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.965471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:21.965471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.965471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:21.965471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:21.965471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:21.965471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:21.965471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:21.965471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:21.965887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.966353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:36:21.966353 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.966353 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:21.966353 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.966353 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.966353 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:21.966353 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.966353 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:21.967194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:21.967929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:21.967929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:21.967929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.967929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:21.967929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.967929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.967929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:21.967929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.967929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:21.969880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.970257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.970257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:21.970257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.970257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:21.970640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:21.971044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:21.971564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.971564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:21.971564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.971564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.971564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:21.971564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.971564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:21.974886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:21.975298 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:21.975425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:21.975613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:21.976001 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:21.976256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.976814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 13:36:21.976814 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.976814 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:21.976814 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.976814 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.976814 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:21.976814 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.976814 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:21.977604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:21.978022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:21.978022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:21.978022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.978022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:21.978022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.978022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.978022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:21.978022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.978022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:21.978400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.978400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:21.978400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:21.978400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:21.978400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.978400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:21.978400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:21.978400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:21.978400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:21.978400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.978400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:21.978732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.979146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.979146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:21.979146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.979146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:21.979340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:21.979829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:21.980362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:21.980362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:21.980362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:21.980362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:22.077536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:22.078046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:22.078157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:22.078896 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:22.078896 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:22.079241 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:22.079481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:22.080236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:22.080236 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:22.080236 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:22.080236 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:22.080236 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:22.080236 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:22.080236 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:22.080236 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:22.081784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:22.083195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:22.083661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:22.084552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:22.085230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:22.085230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:22.085230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:26.911416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:26.912120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:26.912338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:26.913882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:26.950820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:26.951288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.951633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:26.951633 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.951633 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:26.951633 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.951633 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.951633 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:26.951633 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.951633 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:26.952426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.952855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:26.953290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:26.953741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.954354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:26.954354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.954354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.954354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:26.954354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.954354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:26.958040 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:26.958591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:26.958689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:26.959859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:26.960510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:26.960510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.961232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:36:26.961232 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.961232 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:26.961232 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.961232 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.961232 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:26.961232 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.961232 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:26.962813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.963315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:26.963680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.963680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:26.966228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.966857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.967586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:26.967586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.967586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:26.967586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:26.967586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:26.967586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:26.967586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:26.967586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:26.967960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.968686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.970722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:26.971084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:26.971600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:26.972332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.973017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:26.975256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:26.975597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:26.975711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:26.975859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:26.976190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:26.976460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.976977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:36:26.976977 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.976977 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:26.976977 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.976977 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.976977 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:26.976977 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.976977 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:26.977819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:26.978277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:26.978277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:26.978277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.978277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:26.978277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.978277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.978277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:26.978277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.978277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:26.978922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.978922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:26.978922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:26.978922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:26.978922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.978922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:26.978922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:26.978922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:26.978922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:26.978922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.978922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:26.979245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.979607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.979607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:26.979607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.979607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:26.979979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:26.980432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:26.980886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:26.980886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:26.980886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:26.980886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:27.078920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:27.079478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:27.079656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:27.079891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:27.080561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:27.080838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:27.080976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:27.081843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:27.081843 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:27.081843 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:27.081843 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:27.081843 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:27.081843 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:27.081843 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:27.081843 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:27.083372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:27.083744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:27.084079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:27.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:27.085447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:27.085939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:27.085939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:31.912402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:31.913328 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:31.913328 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:31.913916 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:31.950832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:31.951129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.952230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:31.952230 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.952230 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:31.952230 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.952230 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.952230 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:31.952230 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.952230 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:31.954250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.954900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:31.955511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.955511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:31.956023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.956791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.957681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:31.957681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.957681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:31.958636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:31.959198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:31.959441 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:31.960713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:31.961123 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:31.961318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.962110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:36:31.962110 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.962110 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:31.962110 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.962110 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.962110 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:31.962110 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.962110 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:31.963677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.964233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:31.964738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.964738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:31.965602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.966440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.967105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:31.967105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.967105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:31.967105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:31.967105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:31.967105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:31.967105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:31.967445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:31.967851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.968677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:36:31.968677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.968677 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:31.968677 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.968677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.968677 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:31.968677 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.968677 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:31.969419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:31.970012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:31.970012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:31.970012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.970012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:31.970012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.970012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.970012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:31.970012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.970012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.970520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:31.971060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:31.971759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:31.972375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.973013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:31.975607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:31.976390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:31.976598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:31.976831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:31.977492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:31.977862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.978800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:36:31.978800 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.978800 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:31.978800 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.978800 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.978800 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:31.978800 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.978800 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:31.980329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:31.980948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:31.980948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:31.980948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.980948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:31.980948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.980948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.980948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:31.980948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.980948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:31.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:31.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:31.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:31.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:31.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:31.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:31.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:31.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:31.982300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.982827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.982827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:31.982827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.982827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:31.984354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:31.985053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:31.985757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:31.985757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:31.985757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:31.985757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:32.079983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:32.080495 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:32.080627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:32.080875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:32.081499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:32.081771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:32.081926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:32.082647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:32.082647 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:32.082647 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:32.082647 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:32.082647 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:32.082647 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:32.082647 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:32.082647 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:32.084244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:32.085055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:32.085804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:32.085884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:32.086941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:32.088121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:32.088121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:36.913023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:36.913610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:36.913896 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:36.915246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:36.947036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:36.947628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.948814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:36.948814 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.948814 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:36.948814 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.948814 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.948814 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:36.948814 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.948814 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:36.949928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.950544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:36.951145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.951145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:36.951448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.952370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.953317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:36.953317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.953317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:36.959182 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:36.959825 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:36.960016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:36.960832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:36.961061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:36.961226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.961870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:36:36.961870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.961870 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:36.961870 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.961870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.961870 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:36.961870 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.961870 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:36.966233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.966701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:36.967243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:36.967779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.968275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:36.968275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.968275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.968275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:36.968275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.968275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:36.968275 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:36.968275 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:36.968275 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:36.968275 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:36.968582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:36.968582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.972407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:36:36.972407 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.972407 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:36.972407 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.972407 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.972407 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:36.972407 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.972407 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:36.973074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:36.973501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:36.973501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:36.973501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.973501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:36.973501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.973501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.973501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:36.973501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.973501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.973785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:36.974123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:36.974472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:36.974843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.975392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:36.976121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:36.976526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:36.976650 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:36.976787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:36.977299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:36.977645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.978340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:36:36.978340 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.978340 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:36.978340 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.978340 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.978340 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:36.978340 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.978340 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:36.979452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:36.979856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:36.979856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:36.979856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.979856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:36.979856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.979856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.979856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:36.979856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.979856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:36.980806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.980806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:36.980806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:36.980806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:36.980806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.980806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:36.980806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:36.980806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:36.980806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:36.980806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.980806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:36.981108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.981476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.981476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:36.981476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.981476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:36.982901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:36.983577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:36.984420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:36.984420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:36.984420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:36.984420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:37.080679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:37.080977 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:37.081072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:37.081306 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:37.082000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:37.082419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:37.082642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:37.083712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:37.083712 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:37.083712 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:37.083712 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:37.083712 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:37.083712 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:37.083712 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:37.083712 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:37.085852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:37.086725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:37.087472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:37.088239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:37.089386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:37.090566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:37.090566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:41.913721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:41.916360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:41.916360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:41.916360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:41.917142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=247 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:36:41.957771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:41.958187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:41.960560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:41.961071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.961496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:41.961496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.961496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.961496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:41.961496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.961496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:41.962405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.962874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.963436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:41.963436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.963436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:41.963795 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:41.964634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:41.964634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:41.965102 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:41.965349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:41.965504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.966601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:41.966601 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.966601 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:41.966601 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.966601 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.966601 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:41.966601 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.966601 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:41.967490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.967910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:41.968311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.968311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:41.968634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:41.970046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.970613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.971176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:41.971176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.971176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:41.971176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:41.971176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:41.971176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:41.971176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:41.971474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.971992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:36:41.971992 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.971992 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:41.971992 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.971992 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.971992 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:41.971992 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.971992 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:41.972848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:41.973167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:41.973167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:41.973167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.973167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:41.973167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.973167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.973167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:41.973167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.973167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.973872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:41.974144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.974432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.974432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:41.974432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.974432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:41.975402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:41.975722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:41.976046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.976046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:41.976046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.976046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.976046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:41.976046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.976046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:41.978513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:41.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:41.979074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:41.979396 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:41.980592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:41.980999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.981972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:36:41.981972 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.981972 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:41.981972 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.981972 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.981972 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:41.981972 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.981972 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:41.984756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:41.985316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:41.985316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:41.985316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.985316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:41.985316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.985316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.985316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:41.985316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.985316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:41.987635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.987635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:41.987635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:41.987635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:41.987635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.987635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:41.987635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:41.987635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:41.987635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:41.987635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.987635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:41.988046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.988489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.988489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:41.988489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.988489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:41.989734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:41.990342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:41.990886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:41.990886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:41.990886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:41.990886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:42.081355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:42.081763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:42.081942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:42.082290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:42.083329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:42.083979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:42.084279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:42.085302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:42.085302 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:42.085302 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:42.085302 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:42.085302 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:42.085302 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:42.085302 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:42.085302 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:42.086887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:42.087945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:42.088656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:42.089424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:42.090354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:42.091136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:42.091136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:46.914998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:46.915670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:46.915864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:46.917091 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:46.954761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:46.955157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.956154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:36:46.956154 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.956154 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:46.956154 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.956154 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.956154 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:46.956154 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.956154 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:46.957406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.957650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:46.957897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.957897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:46.958706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.959953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:46.959953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.959953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:46.964299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:46.964573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:46.964698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:46.965343 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:46.965671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:46.965894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.966544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:46.966544 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.966544 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:46.966544 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.966544 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.966544 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:46.966544 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.966544 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:46.967764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.968856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:46.969743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:46.970103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.970582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:46.970582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.970582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.970582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:46.970582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.970582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:46.970582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:46.970582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:46.970582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:46.970582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:46.970842 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:46.971526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.971692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:36:46.971692 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.971692 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:46.971692 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.971692 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.971692 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:46.971692 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.971692 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:46.972949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:46.973381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:46.973381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:46.973381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.973381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:46.973381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.973381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.973381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:46.973381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.973381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.974329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:46.974579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.974821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.974821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:46.974821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.974821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:46.977465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:46.977759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:46.978089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.978089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:46.978089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.978089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.978089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:46.978089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.978089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:46.978089 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:46.979987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:46.979987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:46.979987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:46.980518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:46.981254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.981779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:36:46.981779 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.981779 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:46.981779 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.981779 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.981779 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:46.981779 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.981779 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:46.982663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:46.983084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:46.983084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:46.983084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.983084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:46.983084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.983084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.983084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:46.983084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.983084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:46.983766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.983766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:46.983766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:46.983766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:46.983766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.983766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:46.983766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:46.983766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:46.983766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:46.983766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.983766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:46.984053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.984409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.984409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:46.984409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.984409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:46.985493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:46.986812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:46.987507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:46.987507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:46.987507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:46.987507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:47.081933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:47.082411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:47.082566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:47.082771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:47.083432 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:47.083833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:47.084063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:47.084885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:47.084885 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:47.084885 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:47.084885 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:47.084885 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:47.084885 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:47.084885 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:47.084885 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:47.086627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:47.087990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:47.088574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:47.089317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:47.090178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:47.090178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:47.090178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:51.915702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:51.916756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:51.916756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:51.917858 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:51.951745 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:51.952299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:36:51.953540 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.953540 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:51.953540 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.953540 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.953540 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:51.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.953540 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:51.954635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.955348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:51.955999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:51.956797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:51.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:51.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.957757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:51.964904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:51.965343 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:51.965528 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:51.966693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:51.967096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:51.967398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.968403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:51.968403 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.968403 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:51.968403 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.968403 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.968403 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:51.968403 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.968403 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:51.970037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.970655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:51.971294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.971294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:51.971999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.972854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.973741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:51.973741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.973741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:51.973741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:51.973741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:51.973741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:51.973741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:51.974529 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:51.975099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.976306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:36:51.976306 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.976306 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:51.976306 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.976306 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.976306 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:51.976306 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.976306 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:51.977533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:51.977969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:51.977969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:51.977969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.977969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:51.977969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.977969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.977969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:51.977969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.977969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.978588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:51.978985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.979354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.979354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:51.979354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.979354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:51.979974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:51.980426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:51.980942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.980942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:51.980942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.980942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.980942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:51.980942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.980942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:51.980942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:51.980942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:51.980942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:51.981274 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:51.981610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:51.982004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.982813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:36:51.982813 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.982813 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:51.982813 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.982813 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.982813 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:51.982813 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.982813 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:51.983689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:51.984076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:51.984076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:51.984076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.984076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:51.984076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.984076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.984076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:51.984076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.984076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:51.984930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.984930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:51.984930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:51.984930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:51.984930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.984930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:51.984930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:51.984930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:51.984930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:51.984930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.984930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:51.985322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.985685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.985685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:51.985685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.985685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:51.986405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:51.987000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:51.987577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:51.987577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:51.987577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:51.987577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:52.082715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:52.083107 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:52.083289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:52.083524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:52.084654 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:52.085254 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:52.085486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:52.086913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:52.086913 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:52.086913 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:52.086913 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:52.086913 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:52.086913 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:52.086913 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:52.086913 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:52.088502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:52.089195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:52.090171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:52.090997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:52.091882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:52.091882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:52.091882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:56.916594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:56.917494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:56.917494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:56.918622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:56.959039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:56.959502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.960364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:36:56.960364 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.960364 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:56.960364 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.960364 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.960364 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:56.960364 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.960364 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:56.961323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.961693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:56.962049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.962049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:56.962425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.962785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.963132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:56.963132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.963132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:56.965510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:56.966070 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:56.966339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:56.967520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:56.967823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:56.968394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.968919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:36:56.968919 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.968919 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:56.968919 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.968919 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.968919 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:56.968919 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.968919 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:56.971047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.971427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:56.971765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.971765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:56.973220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.973790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.974313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:56.974313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.974313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:56.974313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:56.975048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:56.975048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:56.975048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:36:56.975445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:56.975719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.976439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:36:56.976439 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.976439 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:56.976439 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.976439 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.976439 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:56.976439 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.976439 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:56.978386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:56.979234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:56.979934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.980627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:56.981462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:56.982391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:56.983164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.983164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:56.983164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.983164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.983164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:56.983164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.983164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:56.983164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:56.983164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:56.983164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:56.983567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:36:56.983937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:56.984399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:56.984750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:56.985090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:56.985090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:56.985090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.985090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:56.985090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.985090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.985090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:56.985090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.985090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:56.985343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:56.985791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:56.986520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:56.986520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:56.986520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:56.986520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:36:57.083401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:36:57.083747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:36:57.084094 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:36:57.084094 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:36:57.084777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:57.085258 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:36:57.085558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:57.086527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:36:57.086527 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:57.086527 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:36:57.086527 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:36:57.086527 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:57.086527 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:36:57.086527 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:57.086527 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:57.087462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:36:57.088126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:36:57.088808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:36:57.089726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:36:57.090578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:36:57.090578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:36:57.090578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:01.917595 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:01.918572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:01.918572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:01.919733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:01.961050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:01.961511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.962432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:01.962432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.962432 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:01.962432 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.962432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.962432 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:01.962432 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.962432 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:01.963503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.964325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:01.964990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:01.965730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.968061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:01.968061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.968061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.968061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:01.968061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.968061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:01.968061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:01.968061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:01.968061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:01.968061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:01.968673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:01.968673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.968673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:01.968673 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.968673 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:01.968673 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.968673 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.968673 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:01.968673 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.968673 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:01.969769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.970382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:01.970715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.970715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:01.970846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.971454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.972019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:01.972019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.972019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:01.974854 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:01.975280 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:01.975408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:01.975576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:01.975910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:01.976149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.976745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:01.976745 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.976745 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:01.976745 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.976745 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.976745 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:01.976745 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.976745 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:01.977580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:01.978016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:01.978016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:01.978016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.978016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:01.978016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.978016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.978016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:01.978016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.978016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.978380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:01.978734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.979092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.979092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:01.979092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.979092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:01.979529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:01.980088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:01.980732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.980732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:01.980732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.980732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.980732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:01.980732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.980732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:01.980988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:01.981163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:01.981292 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:01.981426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:01.981713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:01.981939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.982447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:37:01.982447 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.982447 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:01.982447 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.982447 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.982447 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:01.982447 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.982447 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:01.983131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:01.983627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:01.983627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:01.983627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.983627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:01.983627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.983627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.983627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:01.983627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.983627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:01.984072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.984072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:01.984072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:01.984072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:01.984072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.984072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:01.984072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:01.984072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:01.984072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:01.984072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.984072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:01.984486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.984885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.984885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:01.984885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.984885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:01.985166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:01.985740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:01.986429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:01.986429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:01.986429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:01.986429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:02.084110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:02.084498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:02.084639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:02.084876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:02.085673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:02.086133 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:02.086403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:02.087201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:02.087201 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:02.087201 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:02.087201 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:02.087201 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:02.087201 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:02.087201 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:02.087201 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:02.088297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:02.088966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:02.089620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:02.090472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:02.091441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:02.091441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:02.091441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:06.918114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:06.918880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:06.918880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:06.920582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:06.958455 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:06.958745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.959399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:06.959399 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.959399 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:06.959399 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.959399 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.959399 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:06.959399 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.959399 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:06.960248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.960817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:06.961390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:06.962477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.963571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:06.963571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.963571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.963571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:06.963571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.963571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:06.966832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:06.967093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:06.967223 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:06.967994 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:06.968286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:06.968468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.969053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:06.969053 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.969053 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:06.969053 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.969053 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.969053 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:06.969053 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.969053 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:06.970007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.970339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:06.970643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.970643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:06.971035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.971435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.971891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:06.971891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.971891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:06.975203 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:06.975416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:06.975513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:06.975656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:06.976129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:06.976391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.976928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:06.976928 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.976928 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:06.976928 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.976928 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.976928 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:06.976928 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.976928 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:06.977761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:06.978465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:06.978465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:06.978465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.978465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:06.978465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.978465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.978465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:06.978465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.978465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.978871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:06.979379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:06.979916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:06.980528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:06.981756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:06.982452 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:06.982452 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:06.982452 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:06.982791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.983275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:37:06.983275 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.983275 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:06.983275 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.983275 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.983275 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:06.983275 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.983275 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:06.984075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:06.984481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:06.984481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:06.984481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.984481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:06.984481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.984481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.984481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:06.984481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.984481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:06.985057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.985057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:06.985057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:06.985057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:06.985057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.985057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:06.985057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:06.985057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:06.985057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:06.985057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.985057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:06.985383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.985748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.985748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:06.985748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.985748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:06.986403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:06.986852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:06.987342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:06.987342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:06.987342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:06.987342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:07.084565 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:07.085066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:07.085263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:07.085548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:07.086150 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:07.086450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:07.086559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:07.087092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:07.087092 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:07.087092 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:07.087092 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:07.087092 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:07.087092 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:07.087092 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:07.087092 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:07.088761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:07.089545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:07.090385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:07.096618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:07.099965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:07.101152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:07.101152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:11.918709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:11.919544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:11.919697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:11.920811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:11.958636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:11.959042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.960082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:11.960082 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.960082 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:11.960082 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.960082 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.960082 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:11.960082 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.960082 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:11.961127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.961568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:11.961927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.961927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:11.965253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.965768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:11.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:11.967524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:11.968227 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:11.968423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:11.969404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:11.969665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:11.969839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.970585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:11.970585 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.970585 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:11.970585 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.970585 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.970585 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:11.970585 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.970585 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:11.972029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.972322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:11.972663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.972663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:11.973129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.973733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.974308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:11.974308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.974308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:11.975479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:11.975698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:11.975782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:11.975885 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:11.976359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:11.976586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.977059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:11.977059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.977059 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:11.977059 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.977059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.977059 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:11.977059 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.977059 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:11.977746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:11.978368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:11.978368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:11.978368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.978368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:11.978368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.978368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.978368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:11.978368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.978368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.978716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:11.979121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.979558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.979558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:11.979558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.979558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:11.979879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:11.980378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:11.980895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.980895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:11.980895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.980895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.980895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:11.980895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.980895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:11.981801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:11.982006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:11.982114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:11.983017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:11.983526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:11.983907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.984603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:37:11.984603 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.984603 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:11.984603 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.984603 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.984603 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:11.984603 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.984603 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:11.985339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:11.985754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:11.985754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:11.985754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.985754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:11.985754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.985754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.985754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:11.985754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.985754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:11.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:11.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:11.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:11.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:11.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:11.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:11.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:11.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:11.986559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.986930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.986930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:11.986930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.986930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:11.987315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:11.987824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:11.988461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:11.988461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:11.988461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:11.988461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:12.085235 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:12.085689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:12.085821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:12.086156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:12.086832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:12.087428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:12.087771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:12.088774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:12.088774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:12.088774 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:12.088774 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:12.088774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:12.088774 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:12.088774 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:12.088774 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:12.090362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:12.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:12.091756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:12.092232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:12.093092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:12.094033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:12.094033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:16.919529 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:16.920244 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:16.920244 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:16.921266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:16.958533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:16.959090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.960162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:16.960162 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.960162 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:16.960162 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.960162 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.960162 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:16.960162 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.960162 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:16.961111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.961767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:16.962413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:16.963101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.963990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:16.963990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.963990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.963990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:16.963990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.963990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:16.968317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:16.968897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:16.969033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:16.969933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:16.970177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:16.970388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.970987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:16.970987 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.970987 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:16.970987 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.970987 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.970987 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:16.970987 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.970987 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:16.972045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.972465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:16.972989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.972989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:16.973276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.973751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:16.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.974449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:16.975857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:16.975857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:16.975857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:16.975857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:16.977235 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:16.977566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.978365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:16.978365 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.978365 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:16.978365 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.978365 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.978365 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:16.978365 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.978365 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:16.979495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:16.980310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:16.980310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:16.980310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.980310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:16.980310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.980310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.980310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:16.980310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.980310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.980819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:16.981486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:16.982283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:16.983047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:16.984081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.984920 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:16.985636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:16.986090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:16.986090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:16.986090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.986090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:16.986090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.986090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.986090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:16.986090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.986090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:16.986398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.986398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:16.986398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:16.986398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:16.986398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.986398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:16.986398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:16.986398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:16.986398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:16.986398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.986398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:16.986755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.987148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.987148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:16.987148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.987148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:16.987761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:16.988276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:16.988872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:16.988872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:16.988872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:16.988872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:17.085831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:17.086392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:17.086392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:17.086392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:17.087041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:17.087598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:17.087707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:17.089303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:17.089303 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:17.089303 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:17.089303 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:17.089303 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:17.089303 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:17.089303 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:17.089303 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:17.090312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:17.090971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:17.091630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:17.092053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:17.092924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:17.093840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:17.093840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:21.920257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:21.921115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:21.921115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:21.921892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:21.965910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:21.966471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.967540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:21.967540 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.967540 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:21.967540 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.967540 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.967540 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:21.967540 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.967540 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:21.968694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.969468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:21.970319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.970319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:21.970319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:21.970802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.971731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.972814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:21.972814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.972814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:21.972814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:21.972814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:21.972814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:21.972814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:21.972814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.973319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:21.973319 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.973319 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:21.973319 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.973319 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.973319 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:21.973319 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.973319 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:21.974996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.975362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:21.975673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.975673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:21.976008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.976477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.976952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:21.976952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.976952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:21.976952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:21.976952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:21.976952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:21.976952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:21.977409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:21.977702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.978483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:21.978483 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.978483 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:21.978483 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.978483 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.978483 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:21.978483 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.978483 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:21.979472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:21.979973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:21.979973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:21.979973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.979973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:21.979973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.979973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.979973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:21.979973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.979973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.980384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:21.980740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.981132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.981132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:21.981132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.981132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:21.981523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:21.981937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:21.982466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.982466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:21.982466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.982466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.982466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:21.982466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.982466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:21.982701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:21.982858 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:21.982978 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:21.983109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:21.983604 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:21.983955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.984627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:37:21.984627 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.984627 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:21.984627 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.984627 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.984627 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:21.984627 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.984627 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:21.985422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:21.986030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:21.986030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:21.986030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.986030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:21.986030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.986030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.986030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:21.986030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.986030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:21.986478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.986478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:21.986478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:21.986478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:21.986478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.986478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:21.986478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:21.986478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:21.986478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:21.986478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.986478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:21.986920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.987424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.987424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:21.987424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.987424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:21.987592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:21.988230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:21.988899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:21.988899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:21.988899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:21.988899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:22.086596 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:22.086928 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:22.087032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:22.087165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:22.087713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:22.088075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:22.088327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:22.089008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:22.089008 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:22.089008 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:22.089008 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:22.089008 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:22.089008 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:22.089008 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:22.089008 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:22.089794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:22.090677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:22.091459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:22.092515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:22.093528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:22.093528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:22.093528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:26.921022 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:26.921954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:26.922174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:26.923772 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:26.969081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:26.969668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.970114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:26.971325 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:26.971816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:26.971816 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.971816 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:26.971816 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.971816 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.971816 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:26.971816 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.971816 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:26.972159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:26.974406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.974925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:26.975418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.975418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:26.975418 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:26.976179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.976713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.977283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:26.977283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.977283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:26.977283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:26.977283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.977283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:26.977573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:26.977573 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.977573 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:26.977573 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.977573 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.977573 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:26.977573 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.977573 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:26.979629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.980260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:26.980797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.980797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:26.980797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:26.982598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.983266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.983980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:26.983980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.983980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:26.983980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:26.983980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:26.983980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:26.983980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:26.983980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:26.984338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:26.984752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:26.985095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.985949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:26.985949 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.985949 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:26.985949 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.985949 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.985949 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:26.985949 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.985949 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:26.986980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:26.987478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:26.987478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:26.987478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.987478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:26.987478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.987478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.987478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:26.987478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.987478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:26.987478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.988452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:26.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.989246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.989246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:26.989246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.989246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:26.989889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:26.990532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.991115 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:26.991908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:26.992370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:26.992370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:26.992370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.992370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:26.992370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.992370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.992370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:26.992370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.992370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:26.992913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.992913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:26.992913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:26.992913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:26.992913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.992913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:26.992913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:26.992913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:26.992913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:26.992913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.992913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:26.993273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.993683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.993683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:26.993683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.993683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:26.994092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:26.994607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:26.995117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:26.995117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:26.995117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:26.995117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:27.087276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:27.087895 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:27.088000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:27.088199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:27.088859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:27.089297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:27.089542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:27.090367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:27.090367 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:27.090367 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:27.090367 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:27.090367 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:27.090367 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:27.090367 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:27.090367 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:27.091512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:27.092071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:27.092584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:27.093297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:27.094067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:27.094067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:27.094067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:31.921800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:31.922797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:31.922797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:31.924106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:31.960089 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:31.960089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.960348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:31.960348 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.960348 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:31.960348 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.960348 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.960348 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:31.960348 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.960348 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:31.961467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.962409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:31.963192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:31.963842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.964379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:31.964379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.964379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.964379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:31.964379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.964379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:31.971704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:31.972334 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:31.972550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:31.974788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:31.974788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:31.974788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.975251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:31.975251 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.975251 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:31.975251 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.975251 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.975251 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:31.975251 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.975251 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:31.976714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.977384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:31.978068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.978068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:31.978609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.979400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.980311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:31.980311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.980311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:31.980311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:31.980311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:31.980311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:31.980311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:31.980871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:31.981124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.981776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:31.981776 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.981776 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:31.981776 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.981776 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.981776 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:31.981776 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.981776 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:31.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:31.983584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:31.983584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:31.983584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.983584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:31.983584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.983584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.983584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:31.983584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.983584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.984460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:31.984806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.985134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.985134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:31.985134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.985134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:31.985134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:31.985808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:31.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:31.986758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.986758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:31.986758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.986758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.986758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:31.986758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.986758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:31.986758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:31.986758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:31.986758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:31.987247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:31.987671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.988497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:31.988497 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.988497 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:31.988497 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.988497 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.988497 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:31.988497 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.988497 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:31.989227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:31.989687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:31.989687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:31.989687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.989687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:31.989687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.989687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.989687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:31.989687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.989687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:31.990611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.990611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:31.990611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:31.990611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:31.990611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.990611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:31.990611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:31.990611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:31.990611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:31.990611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.990611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:31.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:31.991423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:31.991794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:31.992303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:31.992303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:31.992303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:31.992303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:31.992303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:31.992303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:31.992303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:31.992303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:31.992303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:31.992303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:31.992303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:32.088094 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:32.088353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:32.088562 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:32.088778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:32.089558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:32.089958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:32.090193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:32.091041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:32.091041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:32.091041 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:32.091041 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:32.091041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:32.091041 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:32.091041 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:32.091041 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:32.092699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:32.093343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:32.093990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:32.094332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:32.095047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:32.095994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:32.095994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:36.922412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:36.923384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:36.923607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:36.924960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:36.965810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:36.966309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.967179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:36.967179 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.967179 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:36.967179 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.967179 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.967179 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:36.967179 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.967179 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:36.968593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.969853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:36.970686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:36.971103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.971707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:36.971707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.971707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.971707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:36.971707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.971707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:36.972273 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:36.972914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:36.973101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:36.974110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:36.974365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:36.974692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.975249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:36.975249 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.975249 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:36.975249 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.975249 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.975249 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:36.975249 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.975249 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:36.976719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.977344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:36.978028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:36.979445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.980166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:36.980166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.980166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.980166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:36.980166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.980166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:36.980166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:36.980166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:36.980166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:36.980166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:36.981501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:36.981501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.981501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:36.981501 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.981501 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:36.981501 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.981501 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.981501 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:36.981501 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.981501 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:36.982042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:36.982683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:36.983668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.984558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:36.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:36.985092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:36.985600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.985600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:36.985600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.985600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.985600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:36.985600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.985600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:36.986279 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:36.986814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:36.987113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:36.987446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:36.988125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:36.988509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.989232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:37:36.989232 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.989232 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:36.989232 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.989232 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.989232 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:36.989232 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.989232 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:36.990688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:36.991229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:36.991229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:36.991229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.991229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:36.991229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.991229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.991229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:36.991229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.991229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:36.992474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.992474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:36.992474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:36.992474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:36.992474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.992474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:36.992474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:36.992474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:36.992474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:36.992474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.992474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:36.992824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.993333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.993333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:36.993333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.993333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:36.994769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:36.995329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:36.995962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:36.995962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:36.995962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:36.995962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:37.088746 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:37.089224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:37.089397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:37.089623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:37.090490 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:37.090973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:37.091236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:37.092047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:37.092047 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:37.092047 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:37.092047 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:37.092047 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:37.092047 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:37.092047 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:37.092047 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:37.093577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:37.094173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:37.094808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:37.095201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:37.096157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:37.096940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:37.096940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:41.923255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:41.925037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:41.925037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:41.925310 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:41.927354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:37:41.968728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:41.968975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.969396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:41.969396 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.969396 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:41.969396 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.969396 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.969396 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:41.969396 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.969396 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:41.970409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.971031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:41.971781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:41.972902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.973781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:41.973781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.973781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.973781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:41.973781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.973781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:41.973781 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:41.973781 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:41.973781 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:41.974151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:41.974364 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:41.974513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.974990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:41.974990 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.974990 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:41.974990 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.974990 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.974990 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:41.974990 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.974990 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:41.975943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.976344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:41.976721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.976721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:41.977149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.977644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.978175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:41.978175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.978175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:41.979980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:41.980518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:41.980712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:41.980924 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:41.981364 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:41.981580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.982147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:41.982147 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.982147 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:41.982147 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.982147 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.982147 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:41.982147 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.982147 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:41.983122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:41.983785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:41.983785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:41.983785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.983785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:41.983785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.983785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.983785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:41.983785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.983785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.984151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:41.984821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:41.985346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:41.985995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:41.986732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:41.987199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:41.987199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:41.987344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:41.987882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:41.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.989179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:37:41.989179 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.989179 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:41.989179 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.989179 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.989179 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:41.989179 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.989179 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:41.990098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:41.990733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:41.990733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:41.990733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.990733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:41.990733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.990733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.990733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:41.990733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.990733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:41.991371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.991371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:41.991371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:41.991371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:41.991371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.991371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:41.991371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:41.991371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:41.991371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:41.991371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.991371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:41.991839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.992376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.992376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:41.992376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.992376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:41.992889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:41.993392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:41.993870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:41.993870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:41.993870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:41.993870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:42.089414 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:42.089633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:42.089832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:42.090138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:42.090974 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:42.091431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:42.091710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:42.092656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:42.092656 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:42.092656 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:42.092656 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:42.092656 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:42.092656 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:42.092656 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:42.092656 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:42.094107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:42.094730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:42.095308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:42.095831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:42.097011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:42.098117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:42.098117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:46.923934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:46.924794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:46.924794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:46.925752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:46.964577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:46.964757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.965303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:46.965303 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.965303 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:46.965303 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.965303 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.965303 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:46.965303 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.965303 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:46.966038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.966369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:46.966656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.966656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:46.966845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.967753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.968508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:46.968508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.968508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:46.973822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:46.974496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:46.974686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:46.975573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:46.975741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:46.975858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.976461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:46.976461 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.976461 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:46.976461 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.976461 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.976461 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:46.976461 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.976461 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:46.978336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.978700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:46.979167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.979167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:46.980366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.980805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.981314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:46.981314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.981314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:46.981314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:46.981314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:46.981477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:46.981871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:46.983518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:46.983641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.984079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:46.984079 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.984079 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:46.984079 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.984079 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.984079 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:46.984079 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.984079 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:46.984894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:46.985923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:46.986834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.987402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:46.987816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:46.988370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:46.988983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.988983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:46.988983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.988983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.988983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:46.988983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.988983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:46.988983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:46.988983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:46.988983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:46.990398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:46.990398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:46.990398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.990398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:37:46.990398 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.990398 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:46.990398 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.990398 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.990398 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:46.990398 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.990398 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:46.991104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:46.992283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:46.992283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:46.992283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.992283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:46.992283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.992283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.992283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:46.992283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.992283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:46.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:46.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:46.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:46.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:46.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:46.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:46.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:46.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:46.993519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.993980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.993980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:46.993980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.993980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:46.994283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:46.995640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:46.996262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:46.996262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:46.996262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:46.996262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:47.090021 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:47.090515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:47.090608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:47.090811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:47.091503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:47.091889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:47.092105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:47.093971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:47.093971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:47.093971 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:47.093971 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:47.093971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:47.093971 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:47.093971 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:47.093971 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:47.095027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:47.095625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:47.096193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:47.096931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:47.097665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:47.097665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:47.097665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:51.924602 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:51.925249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:51.925476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:51.926692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:51.962769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:51.963198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.964190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:51.964190 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.964190 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:51.964190 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.964190 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.964190 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:51.964190 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.964190 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:51.965453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.965969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:51.966461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.966461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:51.966861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.967336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.967780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:51.967780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.967780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:51.974371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:51.974785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:51.974968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:51.975757 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:51.975971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:51.976123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.976786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:51.976786 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.976786 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:51.976786 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.976786 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.976786 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:51.976786 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.976786 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:51.977953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.978541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:51.979359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.979359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:51.979492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.980083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.980638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:51.980638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.980638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:51.981366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:51.981697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:51.981815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:51.981943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:51.982386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:51.982674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.983138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:37:51.983138 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.983138 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:51.983138 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.983138 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.983138 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:51.983138 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.983138 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:51.983951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:51.984454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:51.984454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:51.984454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.984454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:51.984454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.984454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.984454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:51.984454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.984454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.984966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:51.985426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.985892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.985892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:51.985892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.985892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:51.986009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:51.986454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:51.986969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.986969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:51.986969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.986969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.986969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:51.986969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.986969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:51.987630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:51.988163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:51.988332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:51.988551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:51.988981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:51.989222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.989719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:37:51.989719 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.989719 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:51.989719 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.989719 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.989719 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:51.989719 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.989719 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:51.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:51.991051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:51.991051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:51.991051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.991051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:51.991051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.991051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.991051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:51.991051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.991051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:51.991643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.991643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:51.991643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:51.991643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:51.991643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.991643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:51.991643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:51.991643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:51.991643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:51.991643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.991643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:51.991996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.992460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.992460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:51.992460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.992460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:51.992986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:51.993614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:51.994277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:51.994277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:51.994277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:51.994277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:52.091033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:52.091330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:52.091330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:52.091594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:52.092091 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:52.092573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:52.092772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:52.093551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:52.093551 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:52.093551 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:52.093551 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:52.093551 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:52.093551 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:52.093551 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:52.093551 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:52.094906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:52.095758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:52.096407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:52.096656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:52.097544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:52.098462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:52.098462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:56.925422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:56.926091 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:56.926175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:56.927688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:56.971168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:56.971443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.972109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:37:56.972109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.972109 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:56.972109 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.972109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.972109 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:56.972109 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.972109 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:56.973411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.974065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:56.974709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.974709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:56.974943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.975511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.976085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:56.976085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.976085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:56.976085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:56.976604 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:56.976900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:56.978099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:56.978379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:56.978552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.979376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:56.979376 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.979376 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:56.979376 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.979376 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.979376 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:56.979376 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.979376 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:56.981439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.981818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:56.982174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.982174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:56.982845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.983307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.983867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:56.983867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.983867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:56.983867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:56.983867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:56.983867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:56.983867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:37:56.984372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:56.984638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.985284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:37:56.985284 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.985284 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:56.985284 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.985284 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.985284 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:56.985284 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.985284 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:56.986187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:56.986678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:56.986678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:56.986678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.986678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:56.986678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.986678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.986678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:56.986678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.986678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.987096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:56.987493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.987861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.987861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:56.987861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.987861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:56.988345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:56.988785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:56.989389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.989389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:56.989389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.989389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.989389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:56.989389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.989389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:56.989389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:56.990322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:56.990322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:56.990322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:56.990756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:56.991164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.994326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:37:56.995720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:56.996532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:56.998256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:56.998684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:56.999566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:57.000090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:57.000090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:57.000090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:57.000090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:57.000090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:57.000090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:57.000090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:57.000090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:37:57.091918 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:37:57.092507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:37:57.092608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:37:57.092803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:37:57.094621 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:57.094621 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:37:57.094621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:57.096264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:37:57.096264 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:57.096264 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:37:57.096264 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:37:57.096264 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:57.096264 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:37:57.096264 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:57.096264 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:57.097140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:37:57.097554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:57.097912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:37:57.099100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:37:57.099771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:37:57.100697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:37:57.100697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:01.935962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:01.936803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:01.936803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:01.937259 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:01.970404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:01.970926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.972146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:38:01.972146 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.972146 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:01.972146 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.972146 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.972146 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:01.972146 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.972146 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:01.974310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.975161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:01.976069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:01.977019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.978245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:01.978245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.978245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.978245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:01.978245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.978245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:01.978245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:01.978245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:01.978245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:01.978245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:01.978833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:01.978833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.979770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:38:01.979770 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.979770 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:01.979770 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.979770 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.979770 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:01.979770 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.979770 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:01.981152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.981649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:01.982012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.982012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:01.982853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.983350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.983886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:01.983886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.983886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:01.984113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:01.984577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:01.984791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:01.984952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:01.985548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:01.985879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.986598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:38:01.986598 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.986598 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:01.986598 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.986598 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.986598 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:01.986598 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.986598 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:01.987242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:01.987713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:01.987713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:01.987713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.987713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:01.987713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.987713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.987713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:01.987713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.987713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.988147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:01.988493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:01.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:01.990576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:01.991090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:01.991614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.991614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:01.991614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.991614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.991614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:01.991614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.991614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:01.991614 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:01.991877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:01.992184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:01.992586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:01.993258 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:01.993531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.994660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:38:01.994660 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.994660 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:01.994660 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.994660 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.994660 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:01.994660 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.994660 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:01.998341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:01.999600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:02.000325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:02.000904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:02.001455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:02.002486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:02.092688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:02.093221 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:02.093827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:02.094286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:02.095120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:02.095674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:02.095916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:02.096917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:02.096917 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:02.096917 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:02.096917 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:02.096917 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:02.096917 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:02.096917 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:02.096917 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:02.097908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:02.098332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:02.098715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:02.098995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:02.099622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:02.100123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:02.100123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:06.936034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:06.936622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:06.936827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:06.938046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:06.977442 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:06.978003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:06.978144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:06.979383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:06.979649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:06.979819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.980496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:38:06.980496 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:06.980496 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:06.980496 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:06.980496 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:06.980496 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:06.980496 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.980496 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:06.981642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:06.982062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:06.982448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.982448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:06.982759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:06.983198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:06.983737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:06.983737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.983737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:06.983737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:06.983737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.983899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:38:06.983899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:06.983899 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:06.983899 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:06.983899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:06.983899 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:06.983899 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.983899 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:06.985345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:06.985781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:06.986119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.986119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:06.986119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:06.987502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:06.987933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:06.988463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:06.988463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.988463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:06.988463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:06.988463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:06.988463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:06.988671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:06.988842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.989487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:38:06.989487 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:06.989487 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:06.989487 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:06.989487 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:06.989487 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:06.989487 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.989487 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:06.990828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:06.991270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:06.991270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:06.991270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:06.991270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:06.991270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:06.991270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:06.991270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:06.991270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.991270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.992478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:06.992813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:06.993182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:06.993182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:06.993182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.993182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:06.993182 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:06.994645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:06.995044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:06.995575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:06.995575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:06.995575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:06.995575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:06.995575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:06.995575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:06.995575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:06.995575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:06.995575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:06.995575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:07.000663 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:07.000948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.001574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:38:07.001574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:07.001574 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:07.001574 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:07.001574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:07.001574 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:07.001574 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.001574 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:07.002351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:07.002979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:07.002979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:07.002979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:07.002979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:07.002979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:07.002979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:07.002979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:07.002979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.002979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:07.003500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.003500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:07.003500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:07.003500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:07.003500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:07.003500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:07.003500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:07.003500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:07.003500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:07.003500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:07.003500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:07.003953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:07.004494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:07.004494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:07.004494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.004494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:07.005117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:07.005829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:07.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:07.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:07.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:07.093358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:07.093788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:07.093860 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:07.093976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:07.094606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:07.095060 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:07.095418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:07.099246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:07.100036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:07.100694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:07.100694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:07.100694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:07.100694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.100694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:07.102932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:07.103908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:07.105436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:07.105436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:11.936843 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:11.938477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:11.938701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:11.940397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:11.979832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:11.981564 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:11.982407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:11.983866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:11.984521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.986051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:38:11.986051 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:11.986051 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:11.986051 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:11.986051 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:11.986051 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:11.986051 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.986051 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:11.987814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:11.988311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:11.988914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.988914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:11.988914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:11.988914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:11.990057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:11.990778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:11.991522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:11.991522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.991522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:11.991522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:11.991522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:11.991522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.991918 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:11.992035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:11.992509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:11.992686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.993286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:11.994902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:38:11.994902 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:11.994902 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:11.994902 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:11.994902 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:11.994902 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:11.994902 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.994902 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:11.995780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:38:11.995780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:11.995780 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:11.995780 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:11.995780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:11.995780 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:11.995780 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.995780 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:11.996189 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:11.997940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:11.998544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:11.999090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.999090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:11.999641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:12.000237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:12.000237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:12.000237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:12.000237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:12.000237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:12.000237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:12.000237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:12.000237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.000237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:12.000237 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:12.001928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:12.002674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:12.003278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:12.003278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.003278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.003913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:12.004300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:12.004704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:12.004704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:12.004704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.004704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:12.004704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:12.007136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:12.007762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:12.008284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:12.008284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:12.008284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:12.008284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:12.008284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:12.008284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.008284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:12.008284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:12.008651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.009640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:38:12.009640 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:12.009640 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:12.009640 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:12.009640 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:12.009640 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:12.009640 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.009640 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:12.014734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:12.015287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:12.015287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:12.015287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:12.015287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:12.015287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:12.015287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:12.015287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:12.015287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.015287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:12.015738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.015 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.015738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:12.015738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:12.015738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:12.015738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.015738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:12.015738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:12.015738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:12.015738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:12.015738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.015738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:12.016330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:12.017086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:12.017852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:12.018850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:12.018850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:12.018850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:12.018850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:12.018850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:12.018850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:12.018850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:12.018850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:12.018850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:12.018850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.018850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:12.094123 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:12.094744 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:12.094861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:12.094956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:12.095872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:12.096271 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:12.096424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.097162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:12.097162 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:12.097162 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:12.097162 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:12.097162 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:12.097162 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:12.097162 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.097162 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.098127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:12.098795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:12.099466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:12.100452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:12.101614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:12.101614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:12.101614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:16.938404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:16.939421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:16.939516 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:16.941142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:16.982649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:16.983539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:16.983940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:16.984047 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:16.984650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.986041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:38:16.986041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:16.986041 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:16.986041 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:16.986041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:16.986041 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:16.986041 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.986041 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:16.986682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:16.988006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:16.988845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:16.989551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.989551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:16.989551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:16.989743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:16.990618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.991890 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:16.993435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:16.994068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:16.994709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:16.997141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:16.998230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:16.998230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:16.998230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:16.998230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:16.998230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.998230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:16.998230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:16.998230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:16.998230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:16.998230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:16.999378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.999378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:16.999378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:16.999378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:16.999378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:16.999378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Traceback (most recent call last):
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:16.999962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.000463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:17.000826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.000826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:17.000826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:17.000826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:17.000826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.000826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:17.000826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:17.000826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:17.000826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:17.000826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.000826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:17.001076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:17.001394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:17.001394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:17.001394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.001394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.001529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:17.004486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Traceback (most recent call last):
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.004863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:17.005382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:17.005713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:17.005982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.006262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.006563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.007029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:17.094716 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:17.096936 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:17.096936 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:17.096936 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:17.096936 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:17.097365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:17.097669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.098682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:17.098682 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:17.098682 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:17.098682 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:17.098682 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:17.098682 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:17.098682 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.098682 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.102137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:17.102587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:17.102981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:17.103506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:17.104132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:17.104132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:17.104132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:21.941110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:21.943933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:21.944959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:21.946374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:21.984501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:21.988774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:21.989087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:21.992964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:21.993414 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:21.993475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:21.993854 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:21.996338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:21.997587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:38:21.997587 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:21.997587 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:21.997587 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:21.997587 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:21.997587 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:21.997587 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:21.997587 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:21.999272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:21.999723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:22.000183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.000183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:22.000183 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:22.005227 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:22.006770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:22.006882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:22.007573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:22.011496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.012130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.012786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:22.012786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.012786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:22.017048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:22.025438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:22.027727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.035339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Traceback (most recent call last):
Apr 21 13:38:22.035339 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.035339 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:22.035339 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.035339 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.035339 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:22.035339 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.035339 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:22.036186 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:22.039699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:22.040188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:22.040188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:22.040188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.040188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:22.040188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.040188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.040188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:22.040188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.040188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:22.041043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] OVN maintenance task set_fip_distributed_flag failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.045276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:22.045800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.046324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.046324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:22.046324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.046324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:22.047450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:22.047992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:22.048494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:22.049113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:22.049729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:22.049729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:22.049729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.049729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:22.049729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.049729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.049729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:22.049729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.049729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:22.050203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.055165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_ha_failover failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.055165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:22.055165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:22.055165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:22.055165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.055165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:22.055165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:22.055165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:22.055165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:22.055165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.055165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:22.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.056013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.056013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:22.056013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.056013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.057090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:22.057725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.058391 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task check_fdb_aging_settings failed after 0.113 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:22.059728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.060177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:22.060653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.060653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:22.061010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.064669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.065149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:22.065149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.065149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:22.096611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:22.097165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:22.097269 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:22.097528 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:22.098042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:22.098378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:22.098560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.099267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:22.099267 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.099267 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:22.099267 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.099267 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.099267 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:22.099267 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.099267 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.100575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:22.100906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.101187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:22.103419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:22.103800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:22.104384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:22.104384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:26.943900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:26.944993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:26.944993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:26.946390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:26.981229 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:26.981602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:26.989760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Traceback (most recent call last):
Apr 21 13:38:26.989760 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:26.989760 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:26.989760 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:26.989760 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:26.989760 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:26.989760 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:26.989760 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:26.991233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:26.991976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:26.992523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:26.993167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:26.993940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:26.993940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:26.993940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:26.993940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:26.993940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:26.993940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:26.993940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:26.993940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:26.993940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:26.995692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:26.995692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:26.995692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:26.995984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:26.995984 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:26.995984 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:26.995984 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:26.995984 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:26.995984 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:26.995984 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:26.995984 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:26.997979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:26.998729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:26.999453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:26.999453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:26.999453 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.000479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.001548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:27.002586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.002586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:27.002586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:27.002586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:27.002586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:27.002586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:27.002586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.002791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:27.002791 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.002791 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:27.002791 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.002791 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.002791 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:27.002791 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.002791 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:27.003383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.004254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:27.004598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.004919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:27.006304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.006766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:27.008011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:27.008011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:27.008011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:27.008011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:27.008011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:27.008011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.008835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:38:27.008835 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.008835 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:27.008835 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.008835 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.008835 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:27.008835 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.008835 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:27.010234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:27.010624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:27.010624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:27.010624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.010624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:27.010624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.010624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.010624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:27.010624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.010624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:27.012233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.012233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:27.012233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:27.012233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:27.012233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.012233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:27.012233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:27.012233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:27.012233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:27.012233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.012233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:27.012539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.012867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.012867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:27.012867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.012867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.014322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:27.014747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.015188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.015188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:27.015188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.015188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:27.097502 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:27.098225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:27.098428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:27.098685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:27.099636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:27.100116 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:27.100432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.101609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:27.101609 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.101609 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:27.101609 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.101609 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.101609 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:27.101609 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.101609 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.102707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:27.103319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:27.103816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:27.104548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:27.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:27.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:27.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:31.944697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:31.948008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:31.948008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:31.948008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:31.996099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:31.996872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:31.997259 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:32.000522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:32.000801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:32.000982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.001676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:32.003533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:32.003533 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.003533 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:32.003533 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.003533 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.003533 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:32.003533 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.003533 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:32.004302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:32.004550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:32.004822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.007996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.008575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:32.009039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.009039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:32.009039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:32.009039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.010863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.011593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:32.012222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.012222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:32.012222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:38:32.012222 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.012222 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:32.012222 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.012222 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.012222 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:32.012222 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.012222 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:32.012222 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:32.012798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:32.013333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.013333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:32.013333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.013333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:32.013333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.013333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.013333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:32.013333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.013333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:32.013333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:32.013333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.014516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:32.015327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Traceback (most recent call last):
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.015756 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:32.016566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.017071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:32.017772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.017772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:32.017772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.017772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.017772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:32.017772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.017772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:32.017772 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:32.017772 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:32.017772 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:32.018150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task check_fdb_aging_settings failed after 0.069 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.022167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.022718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:32.023076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.023076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:32.024006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.024413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.025120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:32.025120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.025120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:32.025120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:38:32.025120 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.025120 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:32.025120 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.025120 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.025120 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:32.025120 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.025120 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:32.025674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:32.025989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.025989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:32.025989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.025989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:32.025989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.025989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.025989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:32.025989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.025989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:32.027034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task update_ha_failover failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.027034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:32.027034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:32.027034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:32.027034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.027034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:32.027034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:32.027034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:32.027034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:32.027034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.027034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:32.027343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.027627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.027627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:32.027627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.027627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.029076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:32.029570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.029948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.029948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:32.029948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.029948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:32.098276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:32.098988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:32.099170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:32.099407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:32.100682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:32.101524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:32.101898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.103146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:32.103146 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.103146 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:32.103146 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.103146 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.103146 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:32.103146 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.103146 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.104148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:32.104808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:32.105600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:32.106351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:32.107201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:32.107201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:32.107201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:36.946387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:36.947171 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:36.947327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:36.948894 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:36.982424 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:36.982579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:36.983015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:38:36.983015 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:36.983015 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:36.983015 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:36.983015 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:36.983015 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:36.983015 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:36.983015 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:36.983729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:36.984082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:36.984456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:36.984456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:36.984520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:36.984918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:36.985401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:36.985401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:36.985401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:36.997281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:36.997657 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:36.997804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:36.998680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:36.998944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:36.999232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.000005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:38:37.000005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.000005 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:37.000005 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.000005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.000005 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:37.000005 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.000005 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:37.002329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.002989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:37.003551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.003551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:37.003648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.004331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.005083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:37.005083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.005083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:37.005083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:37.005083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:37.005083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:37.005083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:37.008135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.010240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:37.011089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:37.011876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.012625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.013023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.013537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:37.013933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:37.014117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:37.014117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:37.014117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:37.014741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:37.014741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.014928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:38:37.014928 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.014928 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:37.014928 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.014928 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.014928 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:37.014928 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.014928 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:37.015691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:37.016336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:37.016336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:37.016336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.016336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:37.016336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.016336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.016336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:37.016336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.016336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:37.016742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.016742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:37.016742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:37.016742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:37.016742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.016742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:37.016742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:37.016742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:37.016742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:37.016742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.016742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:37.017164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.017426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.017426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:37.017426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.017426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.017768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:37.018570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.019359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.019359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:37.019359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.019359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:37.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:37.099806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:37.100026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:37.100090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:37.100847 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:37.101372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:37.101594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.102421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:37.102421 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.102421 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:37.102421 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.102421 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.102421 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:37.102421 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.102421 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.103183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:37.103932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:37.104452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:37.105118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:37.105865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:37.105865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:37.105865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:41.947586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:41.948562 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:41.948787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:41.950101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:41.952908 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:38:41.982112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:41.982521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:41.982920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:38:41.982920 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:41.982920 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:41.982920 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:41.982920 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:41.982920 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:41.982920 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:41.982920 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:41.983576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:41.984526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:41.985398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:41.986591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:41.987513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:41.987513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:41.987513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:41.987513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:41.987513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:41.987513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:41.998180 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:41.998516 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:41.998643 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:41.999718 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:41.999718 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:41.999909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.001031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:38:42.001031 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.001031 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:42.001031 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.001031 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.001031 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:42.001031 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.001031 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:42.002836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.003550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:42.004121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.004121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:42.007414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.008713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:42.008713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.008713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:42.008713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:42.008713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:42.008713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:42.008713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:42.008713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:42.008997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.009533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.010594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:42.011068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:42.011440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:42.012043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.012573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:42.014226 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:42.014618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:42.014784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:42.014953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:42.015231 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:42.015514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.016059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:38:42.016059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.016059 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:42.016059 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.016059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.016059 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:42.016059 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.016059 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:42.016861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:42.017472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:42.017472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:42.017472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.017472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:42.017472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.017472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.017472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:42.017472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.017472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:42.017823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.017823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:42.017823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:42.017823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:42.017823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.017823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:42.017823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:42.017823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:42.017823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:42.017823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.017823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:42.018297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.018836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.018836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:42.018836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.018836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.019052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:42.019777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.020554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.020554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:42.020554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.020554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:42.100249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:42.100983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:42.101096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:42.101445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:42.102386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:42.102843 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:42.103086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.104001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:42.104001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.104001 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:42.104001 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.104001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.104001 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:42.104001 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.104001 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.105575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:42.106313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.107062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:42.108142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:42.108998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:42.109942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:42.109942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:46.948506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:46.949477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:46.949477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:46.950905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:46.987804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:46.988076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:46.988525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:38:46.988525 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:46.988525 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:46.988525 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:46.988525 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:46.988525 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:46.988525 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:46.988525 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:46.989342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:46.989706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:46.990052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:46.990052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:46.990364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:46.990792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:46.991137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:46.991137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:46.991137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:46.998927 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:46.999430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:46.999595 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:47.001163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:47.001672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:47.002063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.003252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:38:47.003252 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.003252 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:47.003252 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.003252 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.003252 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:47.003252 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.003252 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:47.005384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.006261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:47.007159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.007159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:47.007560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.008144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.008836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:47.008836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.008836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:47.008836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:47.008836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:47.008836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:47.008836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:47.009140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:47.009412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.010128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:38:47.010128 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.010128 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:47.010128 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.010128 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.010128 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:47.010128 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.010128 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:47.011046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:47.011670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:47.011670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:47.011670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.011670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:47.011670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.011670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.011670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:47.011670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.011670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.012182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:47.012708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.013280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.013280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:47.013280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.013280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:47.013627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:47.014073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:47.014649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.014649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:47.014649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.014649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.014649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:47.014649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.014649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:47.014876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:47.015132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:47.015246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:47.015434 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:47.015977 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:47.016421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.017112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:38:47.017112 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.017112 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:47.017112 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.017112 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.017112 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:47.017112 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.017112 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:47.017989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:47.018633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:47.018633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:47.018633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.018633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:47.018633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.018633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.018633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:47.018633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.018633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:47.019258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.019258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:47.019258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:47.019258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:47.019258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.019258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:47.019258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:47.019258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:47.019258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:47.019258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.019258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:47.019817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.020427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.020427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:47.020427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.020427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.020782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:47.021299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.021800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.021800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:47.021800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.021800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:47.101513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:47.102252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:47.102440 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:47.102811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:47.103671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:47.104393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:47.104673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.105614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:47.105614 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.105614 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:47.105614 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.105614 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.105614 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:47.105614 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.105614 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.107582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:47.108392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.109065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:47.109657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:47.110720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:47.111428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:47.111428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:51.949373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:51.950192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:51.950192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:51.950961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:51.985966 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:51.986417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:51.987476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:38:51.987476 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:51.987476 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:51.987476 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:51.987476 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:51.987476 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:51.987476 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:51.987476 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:51.988368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:51.988822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:51.989321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:51.989321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:51.989567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:51.989947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:51.990622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:51.990622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:51.990622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:51.999577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:52.000066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:52.000360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:52.001253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:52.001544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:52.001766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.002599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:38:52.002599 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.002599 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:52.002599 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.002599 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.002599 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:52.002599 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.002599 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:52.004157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.004782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:52.005342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.005342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:52.006031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.006869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.007794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:52.007794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.007794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:52.008420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:52.008775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:52.008864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:52.009013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:52.009362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:52.009549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.010005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:38:52.010005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.010005 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:52.010005 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.010005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.010005 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:52.010005 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.010005 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:52.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:52.011567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:52.011567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:52.011567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.011567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:52.011567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.011567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.011567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:52.011567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.011567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.011938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:52.012351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.012720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.012720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:52.012720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.012720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:52.013011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:52.013476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:52.013935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.013935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:52.013935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.013935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.013935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:52.013935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.013935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:52.015204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:52.015566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:52.015719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:52.015854 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:52.016222 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:52.016531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.017418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:38:52.017418 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.017418 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:52.017418 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.017418 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.017418 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:52.017418 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.017418 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:52.018131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:52.018589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:52.018589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:52.018589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.018589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:52.018589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.018589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.018589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:52.018589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.018589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:52.019081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.019081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:52.019081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:52.019081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:52.019081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.019081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:52.019081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:52.019081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:52.019081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:52.019081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.019081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:52.019414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.019781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.019781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:52.019781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.019781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.020232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:52.020867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.021622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.021622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:52.021622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.021622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:52.102198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:52.102493 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:52.102598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:52.102819 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:52.103553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:52.103994 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:52.104299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.105309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:52.105309 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.105309 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:52.105309 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.105309 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.105309 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:52.105309 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.105309 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.106227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:52.106610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:52.107577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:52.108054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:52.108575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:52.108575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:56.950228 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:56.951015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:56.951230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:56.952580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:56.986595 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:56.987129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:56.988155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:38:56.988155 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:56.988155 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:56.988155 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:56.988155 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:56.988155 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:56.988155 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:56.988155 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:56.988978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:56.989691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:56.990378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:56.991114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:56.992165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:56.992165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:56.992165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:56.992165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:56.992165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:56.992165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:57.001520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:57.002097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:57.002224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:57.003075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:57.003417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:57.003652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.004604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:38:57.004604 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.004604 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:57.004604 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.004604 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.004604 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:57.004604 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.004604 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:57.005530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.006144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:57.006916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:57.007499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:57.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:57.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.008020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:57.008972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:57.009373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:57.009474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:57.009603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:38:57.009904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:57.010100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.010582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:38:57.010582 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.010582 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:57.010582 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.010582 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.010582 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:57.010582 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.010582 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:57.011329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:57.011899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:57.011899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:57.011899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.011899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:57.011899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.011899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.011899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:57.011899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.011899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.012359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:57.012718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:57.013193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:57.013733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.014415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:57.015658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:57.015943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:57.016054 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:57.016266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:57.016673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:57.017044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.017748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:38:57.017748 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.017748 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:57.017748 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.017748 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.017748 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:57.017748 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.017748 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:38:57.018466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:38:57.018952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:57.018952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:38:57.018952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.018952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:38:57.018952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.018952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.018952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:38:57.018952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.018952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:38:57.019357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.019357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:57.019357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:57.019357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:57.019357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.019357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:57.019357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:57.019357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:57.019357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:57.019357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.019357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:57.019668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.020191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.020191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:57.020191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.020191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.020493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:57.020962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.021480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.021480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:57.021480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.021480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:38:57.103094 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:38:57.103822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:38:57.103949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:38:57.104170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:38:57.104662 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:57.105030 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:38:57.105232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.105948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:38:57.105948 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.105948 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:38:57.105948 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.105948 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.105948 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:38:57.105948 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.105948 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.107016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:38:57.107767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:38:57.108463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:38:57.109454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:38:57.109964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:38:57.109964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:38:57.109964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:01.950898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:01.952006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:01.952006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:01.952918 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:01.985241 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:01.985637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:01.986447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:39:01.986447 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:01.986447 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:01.986447 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:01.986447 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:01.986447 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:01.986447 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:01.986447 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:01.987240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:01.987654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:01.988084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:01.988577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:01.989233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:01.989233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:01.989233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:01.989233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:01.989233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:01.989233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:02.002288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:02.003184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:02.003184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:02.003471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:02.003666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:02.003806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:02.006502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:02.007142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.007559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:02.007998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:02.008633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:02.008633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:02.008633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:02.008633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.008633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:02.008633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.008633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.008633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:02.008633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.008633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:02.009386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:02.009713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:02.009947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:02.010162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:02.010624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:02.010809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.014246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:02.015070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:02.015751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.016273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:02.016741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.016741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:02.017689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:02.018404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:02.018908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.018908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:02.018908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.018908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.018908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:02.018908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.018908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:02.019438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:02.019851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:02.020481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:02.020481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:02.020642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:02.020886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.023495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:02.023843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:02.024188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.024446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:02.024685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:02.024973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:02.025377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:02.025377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.025377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:02.025377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.025377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.025377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:02.025377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.025377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:02.103913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:02.104446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:02.104607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:02.104859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:02.105366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:02.105831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:02.106084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.107811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:39:02.107811 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.107811 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:02.107811 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.107811 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.107811 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:02.107811 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.107811 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.109326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:02.109919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.110464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:02.110984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:02.111710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:02.112528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:02.112528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:06.951618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:06.952346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:06.952346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:06.953051 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:06.991049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:06.991500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:06.992408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:39:06.992408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:06.992408 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:06.992408 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:06.992408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:06.992408 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:06.992408 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:06.992408 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:06.993154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:06.993780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:06.994501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:06.995131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:06.996051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:06.996051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:06.996051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:06.996051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:06.996051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:06.996051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:07.002843 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:07.003831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:07.003831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:07.004463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:07.004810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:07.005063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.006441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:07.006441 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.006441 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:07.006441 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.006441 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.006441 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:07.006441 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.006441 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:07.007766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.008546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:07.009041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.009041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:07.010318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.010957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.011667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:07.011667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.011667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:07.011667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:07.011667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:07.011667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:07.011667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:07.011667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:07.012000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.012000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:39:07.012000 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.012000 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:07.012000 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.012000 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.012000 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:07.012000 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.012000 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:07.013101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:07.015655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:07.016243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.016823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:07.017291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:07.018695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:07.019370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.019370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:07.019370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.019370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.019370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:07.019370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.019370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:07.019912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:07.020195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:07.020342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:07.020470 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:07.021053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:07.021549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.022282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:39:07.022282 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.022282 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:07.022282 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.022282 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.022282 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:07.022282 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.022282 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:07.023002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:07.023454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:07.023454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:07.023454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.023454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:07.023454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.023454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.023454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:07.023454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.023454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:07.023858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.023858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:07.023858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:07.023858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:07.023858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.023858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:07.023858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:07.023858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:07.023858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:07.023858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.023858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:07.024347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:07.025107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:07.025534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:07.026005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:07.026005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:07.026005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:07.026005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:07.026005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.026005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:07.026005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.026005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.026005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:07.026005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.026005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:07.104699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:07.105108 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:07.105309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:07.105539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:07.106155 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:07.106678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:07.106908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.107680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:39:07.107680 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.107680 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:07.107680 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.107680 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.107680 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:07.107680 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.107680 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.108727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:07.109299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.110047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:07.110332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:07.110883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:07.111416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:07.111416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:11.952382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:11.952948 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:11.953058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:11.954484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:11.987334 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:11.987743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:11.988704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:39:11.988704 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:11.988704 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:11.988704 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:11.988704 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:11.988704 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:11.988704 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:11.988704 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:11.989764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:11.990678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:11.991578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:11.992614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:11.993869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:11.993869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:11.993869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:11.993869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:11.993869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:11.993869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:12.003501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:12.004151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:12.004506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:12.005762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:12.006188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:12.006481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.007381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:12.007381 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.007381 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:12.007381 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.007381 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.007381 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:12.007381 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.007381 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:12.009123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.010018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:12.010886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.010886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:12.011093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.011948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.012675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:12.012675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.012675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:12.012675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:12.012675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:12.012675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:12.012675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:12.013018 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:12.013947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.014841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:12.014841 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.014841 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:12.014841 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.014841 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.014841 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:12.014841 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.014841 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:12.015592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:12.016302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:12.016302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:12.016302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.016302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:12.016302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.016302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.016302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:12.016302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.016302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:12.017165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:12.017705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:12.018251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.019017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:12.020229 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:12.020518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:12.020627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:12.020758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:12.021134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:12.021363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.021953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:39:12.021953 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.021953 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:12.021953 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.021953 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.021953 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:12.021953 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.021953 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:12.022747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:12.023463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:12.023463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:12.023463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.023463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:12.023463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.023463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.023463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:12.023463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.023463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:12.024335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.024335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:12.024335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:12.024335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:12.024335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.024335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:12.024335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:12.024335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:12.024335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:12.024335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.024335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:12.024837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.025370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.025370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:12.025370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.025370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.026315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:12.027916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.028409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.028409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:12.028409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.028409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:12.105287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:12.106195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:12.106195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:12.106195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:12.106566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:12.106916 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:12.107116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.107819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:39:12.107819 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.107819 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:12.107819 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.107819 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.107819 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:12.107819 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.107819 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.108767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:12.109342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.109707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:12.109905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:12.110499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:12.111051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:12.111051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:16.952946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:16.953657 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:16.953812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:16.954630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:16.992465 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:16.992465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:16.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:39:16.992968 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:16.992968 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:16.992968 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:16.992968 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:16.992968 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:16.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:16.992968 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:16.994016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:16.994934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:16.995713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:16.996616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:16.997938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:16.997938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:16.997938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:16.997938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:16.997938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:16.997938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:17.004247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:17.004786 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:17.004973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:17.005975 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:17.006462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:17.006781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.007833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:39:17.007833 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.007833 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:17.007833 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.007833 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.007833 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:17.007833 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.007833 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:17.009438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.009810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:17.010173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.010173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:17.011506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.012045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.012613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:17.012613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.012613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:17.012613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:17.012613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:17.012889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:17.013143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:17.013617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:17.013790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.014462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:17.014462 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.014462 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:17.014462 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.014462 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.014462 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:17.014462 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.014462 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:17.015703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:17.016292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:17.016292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:17.016292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.016292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:17.016292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.016292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.016292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:17.016292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.016292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.016731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:17.017160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.017599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.017599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:17.017599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.017599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:17.017971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:17.018519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:17.019163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.019163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:17.019163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.019163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.019163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:17.019163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.019163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:17.020938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:17.021359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:17.021535 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:17.021816 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:17.022229 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:17.022523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.023015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:39:17.023015 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.023015 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:17.023015 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.023015 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.023015 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:17.023015 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.023015 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:17.023866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:17.024771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:17.025791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:17.026579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:17.027482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.028311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:17.105951 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:17.106446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:17.106589 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:17.106673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:17.107051 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:17.107541 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:17.107763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.108655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:39:17.108655 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.108655 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:17.108655 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.108655 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.108655 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:17.108655 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.108655 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.109918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:17.110673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.111325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:17.111690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:17.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:17.113350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:17.113350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:21.953612 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:21.954481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:21.954481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:21.955635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:22.005691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:22.006394 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:22.006554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:22.007925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:22.008327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:22.008572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.009474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:39:22.009474 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.009474 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:22.009474 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.009474 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.009474 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:22.009474 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.009474 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.010609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.011099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:22.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:22.012061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.012508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.012971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:22.012971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.012971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:22.012971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:22.013171 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:22.013557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.014266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:22.014939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:39:22.014939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.014939 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:22.014939 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.014939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.014939 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:22.014939 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.014939 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:22.015259 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.016237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.016675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:22.017126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.017126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:22.017126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:22.018292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.018774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.019360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:22.019360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.019360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:22.019360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:22.019360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.019502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:22.019502 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.019502 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:22.019502 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.019502 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.019502 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:22.019502 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.019502 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:22.020434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:22.020840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.020840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:22.020840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.020840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:22.020840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.020840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.020840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:22.020840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.020840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.022103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:22.022485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.022843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.022843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:22.022843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.022843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:22.022843 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:22.024433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.024849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:22.025331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.025331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:22.025331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.025331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.025331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:22.025331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.025331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:22.025331 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:22.025331 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:22.025331 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:22.025595 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:22.025823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.026362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:39:22.026362 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.026362 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:22.026362 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.026362 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.026362 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:22.026362 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.026362 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:22.027330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:22.028071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.028071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:22.028071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.028071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:22.028071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.028071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.028071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:22.028071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.028071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:22.028527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.028527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:22.028527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:22.028527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:22.028527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.028527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:22.028527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:22.028527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:22.028527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:22.028527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.028527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:22.029023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:22.029654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:22.030319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.031181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:22.031181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:22.031181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.031181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:22.031181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.031181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:22.031181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.031181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.031181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:22.031181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.031181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:22.106600 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:22.107132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:22.107395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:22.107643 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:22.108163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:22.108695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:22.108985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.109906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:39:22.109906 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.109906 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:22.109906 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.109906 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.109906 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:22.109906 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.109906 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.111394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:22.112341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.113068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:22.114985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:22.115902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:22.116790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:22.116790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:26.954492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:26.956728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:26.956840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:26.957926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:26.992762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:26.993034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:26.993767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:39:26.993767 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:26.993767 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:26.993767 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:26.993767 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:26.993767 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:26.993767 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:26.993767 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:26.994610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:26.994984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:26.995370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:26.995370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:26.995554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:26.996091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:26.996717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:26.996717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:26.996717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:27.006686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:27.008042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:27.008042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:27.008361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:27.008612 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:27.008778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.009732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:39:27.009732 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.009732 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:27.009732 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.009732 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.009732 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:27.009732 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.009732 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:27.012838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.013498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:27.013898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:27.014384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.014958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:27.014958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.014958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.014958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:27.014958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.014958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:27.014958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:27.014958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:27.014958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:27.015292 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:27.015915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:27.016351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.017717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:27.017717 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.017717 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:27.017717 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.017717 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.017717 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:27.017717 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.017717 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:27.018691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:27.019229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:27.019229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:27.019229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.019229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:27.019229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.019229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.019229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:27.019229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.019229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.019633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:27.020393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:27.021153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:27.021874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.022464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:27.022983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:27.023491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:27.023559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:27.023693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:27.024272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:27.024784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.025466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:39:27.025466 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.025466 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:27.025466 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.025466 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.025466 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:27.025466 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.025466 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:27.026176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:27.026671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:27.026671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:27.026671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.026671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:27.026671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.026671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.026671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:27.026671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.026671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:27.026947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.026947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:27.026947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:27.026947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:27.026947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.026947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:27.026947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:27.026947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:27.026947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:27.026947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.026947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:27.027329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:27.027751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:27.028196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:27.028832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:27.028832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:27.028832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:27.028832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:27.028832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.028832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:27.028832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.028832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.028832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:27.028832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.028832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:27.107234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:27.107695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:27.107848 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:27.108062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:27.108749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:27.109406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:27.109875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.111499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:39:27.111499 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.111499 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:27.111499 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.111499 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.111499 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:27.111499 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.111499 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.113472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:27.113815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.114060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:27.115400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:27.116182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:27.116991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:27.116991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:31.956869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:31.957691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:31.957826 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:31.958849 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:31.998351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:31.998880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.000239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:39:32.000239 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.000239 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:32.000239 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.000239 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.000239 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:32.000239 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.000239 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.001143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.001974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:32.002575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:32.003168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.004145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:32.004145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.004145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.004145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:32.004145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.004145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:32.007250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:32.007975 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:32.008129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:32.009302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:32.009549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:32.009685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.010427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:39:32.010427 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.010427 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:32.010427 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.010427 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.010427 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:32.010427 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.010427 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.012598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.013124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:32.013850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.013850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:32.014820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.015618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.016399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:32.016399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.016399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:32.016399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:32.016399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:32.016399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:32.016399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:32.017408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:32.017657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.018546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:32.018546 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.018546 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:32.018546 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.018546 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.018546 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:32.018546 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.018546 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:32.019519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:32.019975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.019975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:32.019975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.019975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:32.019975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.019975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.019975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:32.019975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.019975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.020532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:32.020934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.021365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.021365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:32.021365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.021365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:32.021778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.022269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:32.022714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.022714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:32.022714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.022714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.022714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:32.022714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.022714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:32.023404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:32.023678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:32.023801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:32.023937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:32.026548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:32.026996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.028441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:39:32.028441 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.028441 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:32.028441 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.028441 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.028441 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:32.028441 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.028441 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:32.029228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:32.029621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.029621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:32.029621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.029621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:32.029621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.029621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.029621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:32.029621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.029621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:32.030200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.030200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:32.030200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:32.030200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:32.030200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.030200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:32.030200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:32.030200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:32.030200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:32.030200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.030200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:32.030465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.030791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.030791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:32.030791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.030791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.031629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:32.032072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.032636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.032636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:32.032636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.032636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:32.108076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:32.109146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:32.109146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:32.109339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:32.109958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:32.110456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:32.110756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.111990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:39:32.111990 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.111990 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:32.111990 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.111990 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.111990 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:32.111990 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.111990 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.113475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:32.113914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.114279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:32.115545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:32.116045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:32.116598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:32.116598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:36.974620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:36.978988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:36.978988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:36.978988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:37.006790 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:37.007329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.008614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:39:37.008614 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.008614 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:37.008614 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.008614 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.008614 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:37.008614 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.008614 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.010555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.011364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:37.012036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.012036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:37.012036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:37.013195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.013972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.014625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:37.014625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.014625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:37.014625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:37.014625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:37.014884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:37.015633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:37.015633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.015767 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:37.016153 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:37.016253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:37.016418 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:37.017431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:39:37.017431 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.017431 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:37.017431 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.017431 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.017431 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:37.017431 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.017431 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.018693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.019108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:37.019505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.019505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:37.019505 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:37.020849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.021497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.022179 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:37.023105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:37.023851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.023851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:37.023851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.023851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:37.023851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.023851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.023851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:37.023851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.023851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.024374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:37.024950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.025558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.025558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:37.025558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.025558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:37.026195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.026877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:37.027651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.027651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:37.027651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.027651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.027651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:37.027651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.027651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:37.027651 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:37.027651 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:37.027651 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:37.028144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:37.028144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:37.028144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.028766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:37.028766 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.028766 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:37.028766 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.028766 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.028766 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:37.028766 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.028766 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:37.029696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:37.030021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.030021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:37.030021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.030021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:37.030021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.030021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.030021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:37.030021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.030021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:37.030741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.030741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:37.030741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:37.030741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:37.030741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.030741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:37.030741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:37.030741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:37.030741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:37.030741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.030741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:37.030960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.031233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.031233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:37.031233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.031233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.031962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:37.032379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.032739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.032739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:37.032739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.032739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:37.108999 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:37.109809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:37.109925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:37.110408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:37.111051 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:37.111783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:37.112423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.113666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:39:37.113666 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.113666 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:37.113666 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.113666 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.113666 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:37.113666 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.113666 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.114641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:37.115440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:37.115976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:37.116684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:37.117475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:37.117475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:37.117475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:41.975548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:41.976326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:41.976567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:41.978288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:41.980107 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:39:42.012096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:42.012512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.013506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:39:42.013506 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.013506 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:42.013506 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.013506 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.013506 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:42.013506 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.013506 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.015178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.015859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:42.016202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.016202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:42.016788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.017280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.017739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:42.017739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.017739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:42.017739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:42.017739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:42.017739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:42.017739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:42.018056 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:42.018599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:42.018823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:42.019415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:42.020112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:42.020112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.020995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:39:42.020995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.020995 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:42.020995 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.020995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.020995 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:42.020995 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.020995 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.023108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.023518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:42.023850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.023850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:42.023850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:42.025509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.026136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:42.027699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.028488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task set_fip_distributed_flag failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:42.029365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:42.029735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.030186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.030675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:42.031341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:42.031792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:42.031792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:42.031792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:42.031792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:42.031792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.031950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:39:42.031950 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.031950 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:42.031950 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.031950 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.031950 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:42.031950 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.031950 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:42.033834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:42.034398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.034398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:42.034398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.034398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:42.034398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.034398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.034398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:42.034398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.034398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:42.036387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.036387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:42.036387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:42.036387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:42.036387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.036387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:42.036387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:42.036387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:42.036387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:42.036387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.036387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:42.036756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.037118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.037118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:42.037118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.037118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.038799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:42.039540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.040049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.040049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:42.040049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.040049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:42.110139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:42.110566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:42.110641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:42.110836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:42.111355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:42.111619 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:42.111750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.112268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:39:42.112268 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.112268 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:42.112268 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.112268 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.112268 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:42.112268 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.112268 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.113279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:42.114042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:42.114711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:42.115269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:42.115794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:42.115794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:42.115794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:46.458783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:46.459777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:46.459777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:46.470149 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 21 13:39:46.470290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, security_groups=2, routers=1, router_ports=1 {{(pid=106267) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 21 13:39:46.470389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Fixing resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:39:46.505000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, resource={'id': 'b57c058f-054e-4a2a-980f-6bfda2111e90', 'name': 'private', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['6a3993e6-b5ba-4812-baa3-b25db5880cd9', 'c97deb27-a1d6-4e03-b489-95abde8fd791'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:29:32Z', 'updated_at': '2026-04-21T13:29:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 24380}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.505613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.506235 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3f6c78fa-b671-412d-bb34-5426a40d8bff, external_ids_update=None, columns={'tag': 24380}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.506585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.508300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:39:46.508300 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:46.508300 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:46.508300 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:46.508300 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:46.508300 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:46.508300 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.508300 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:46.509780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Failed to fix resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.509780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:46.509780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:39:46.509780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:39:46.509780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:39:46.509780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:39:46.509780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:39:46.509780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:46.509780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:46.509780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:46.509780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:46.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:46.511315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:46.511315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.511315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:46.511315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Fixing resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:39:46.541457 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d0607710-681d-444d-be26-dbeca5a07b46, resource={'id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'name': 'public', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-21T13:29:38Z', 'updated_at': '2026-04-21T13:29:48Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.541891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d0607710-681d-444d-be26-dbeca5a07b46, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.542539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-19f04b40-ed99-482a-8774-a76892a716d1, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.542884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.544059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:39:46.544059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:46.544059 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:46.544059 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:46.544059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:46.544059 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:46.544059 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.544059 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:46.545292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Failed to fix resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.545292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:46.545292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:39:46.545292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:39:46.545292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:39:46.545292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:39:46.545292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:39:46.545292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:46.545292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:46.545292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:46.545292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:46.545904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:46.546648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:46.546648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.546648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:46.546648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Fixing resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:39:46.579952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, resource={'id': '345e31fe-ede8-4dd0-823e-e81dcbc5bef0', 'name': 'admin_net', 'tenant_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'project_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['bb271c4b-4172-418f-995c-f72abd439dbe'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:34:46Z', 'updated_at': '2026-04-21T13:34:47Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 22486}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.580566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.581200 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-c1b4e94e-49ea-4c1f-8edf-2f1433fdff16, external_ids_update=None, columns={'tag': 22486}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.581473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.583014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:39:46.583014 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:46.583014 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:46.583014 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:46.583014 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:46.583014 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:46.583014 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.583014 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:46.585469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Failed to fix resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.585469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:46.585469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:39:46.585469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:39:46.585469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:39:46.585469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:39:46.585469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:39:46.585469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:46.585469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:46.585469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:46.585469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:46.585960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:46.586518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:46.586518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.586518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:46.586518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Fixing resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:39:46.622570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, resource={'id': 'b364ad09-f70d-45d8-b9dd-8420f5f1c377', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['620e9559-2c15-4370-904f-f4677bc69542'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:35:24Z', 'updated_at': '2026-04-21T13:35:25Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 61006}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.622928 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.623407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a100d4e8-7f11-40c9-8e4a-346c2c764f69, external_ids_update=None, columns={'tag': 61006}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.623613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.624784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:39:46.624784 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:46.624784 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:46.624784 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:46.624784 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:46.624784 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:46.624784 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.624784 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:46.626142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Failed to fix resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.626142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:46.626142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:39:46.626142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:39:46.626142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:39:46.626142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:39:46.626142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:39:46.626142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:46.626142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:46.626142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:46.626142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:46.626928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:46.627973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:46.627973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.627973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:46.627973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Fixing resource 4b2f2001-2761-4de7-82db-4f6fb80c4f9d (type: security_groups) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:39:46.683194 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.db.ovn_revision_numbers_db [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Successfully bumped revision number for resource 4b2f2001-2761-4de7-82db-4f6fb80c4f9d (type: security_groups) to 15
Apr 21 13:39:46.683433 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Fixing resource ff426b81-519a-4ff9-9873-ed6df0f04b61 (type: security_groups) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:39:46.725446 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.db.ovn_revision_numbers_db [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Successfully bumped revision number for resource ff426b81-519a-4ff9-9873-ed6df0f04b61 (type: security_groups) to 10
Apr 21 13:39:46.725597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Fixing resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:39:46.831444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, resource={'id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'enable_snat': True}, 'gw_port_id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-61m1f', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'ip_address': '172.24.5.49', 'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'}, {'ip_address': '2001:db8::162', 'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 3, 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04'}, resource_type=routers, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.831926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.832291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.832672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:46.832784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.835156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:39:46.835156 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:46.835156 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:46.835156 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:46.835156 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:46.835156 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:46.835156 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.835156 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:46.835479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Unable to update router 06899d4c-a436-45e4-bed5-95dabe888f51. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.837791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Failed to fix resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.837791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:46.837791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:39:46.837791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:39:46.837791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:39:46.837791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:39:46.837791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 21 13:39:46.837791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 21 13:39:46.837791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 21 13:39:46.837791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 21 13:39:46.837791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:46.838188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:46.838598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:46.838598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:46.838598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:46.838598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:46.838598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:46.838598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:46.838598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:46.838598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:46.838598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:46.838598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Fixing resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:39:46.976880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:46.978105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:46.979486 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:46.983134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:47.017891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:47.018507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:47.019946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:47.027062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:47.030531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:47.034498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:47.035234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:47.039050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:47.039443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:47.042519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:47.051081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:47.052422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.059552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:47.060105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:47.063091 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:47.064822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.071543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:39:47.071543 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.071543 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:47.071543 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.071543 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.071543 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:47.071543 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.071543 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:47.072745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:39:47.072745 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.072745 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:47.072745 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.072745 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.072745 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:47.072745 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.072745 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.082893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.083684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:47.084301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.084301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:47.090523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:47.091329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.091329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:47.091329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.091329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:47.091329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.091329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.091329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:47.091329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.091329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.093473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:47.094126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.094734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.094734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:47.094734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.094734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:47.095604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.096464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.097371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:47.097371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.097371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:47.097371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:47.098087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.098843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:47.099624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.099624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:47.099624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.099624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.099624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:47.099624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.099624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:47.099624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.099971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 13:39:47.099971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.099971 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:47.099971 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.099971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.099971 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:47.099971 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.099971 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:47.103370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:47.104017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.104017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:47.104017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.104017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:47.104017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.104017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.104017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:47.104017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.104017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:47.107039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_ha_failover failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.107039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:47.107039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:47.107039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:47.107039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.107039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:47.107039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:47.107039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:47.107039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:47.107039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.107039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:47.107433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.107835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.107835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:47.107835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.107835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:47.109354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.113380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:47.114192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.115039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.115039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:47.115039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.115039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:47.115039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.115039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:47.117309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:39:47.117309 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.117309 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:47.117309 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.117309 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.117309 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:47.117309 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.117309 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:47.117590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, resource={'id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'name': '', 'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'project_id': '', 'mac_address': 'fa:16:3e:fb:71:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000168688', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:47.117590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_fdb_aging_settings failed after 0.136 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.118597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.119530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:47.120456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.120456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:47.120456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3 ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'neutron:network_name': 'neutron-d0607710-681d-444d-be26-dbeca5a07b46', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-06899d4c-a436-45e4-bed5-95dabe888f51'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.49/24', '2001:db8::162/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:47.120456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:47.120456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:47.120959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, lrouter_port=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:47.121819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.122601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.123400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:47.123400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.123400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:47.123400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.123400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:47.123400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:47.123400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.124019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 13:39:47.124019 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.124019 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:47.124019 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.124019 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.124019 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:47.124019 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.124019 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.126197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.126818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:47.127324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.127324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:47.127324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:39:47.127324 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.127324 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:47.127324 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.127324 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.127324 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:47.127324 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.127324 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:47.129168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.129515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.129911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:47.129911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.129911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:47.130351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Maintenance task: Failed to fix resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.130351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:47.130351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:39:47.130351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:39:47.130351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:39:47.130351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:39:47.130351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 21 13:39:47.130351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:47.130351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.130351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.130351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:47.130602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:47.130858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:47.130858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:47.130858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:47.130858 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task check_for_inconsistencies finished in 0.671 seconds
Apr 21 13:39:51.978556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:51.979613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:51.979613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:51.981093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:52.016682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:52.017004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.019061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:39:52.019061 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.019061 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:52.019061 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.019061 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.019061 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:52.019061 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.019061 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.019661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.020386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:52.020990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:52.021763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.023375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:52.023375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.023375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.023375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:52.023375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.023375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:52.023375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:52.023375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:52.023375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:52.023375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:52.023946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:52.023946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:52.023946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:52.023946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:52.024261 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.027241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.027855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:52.028112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.028112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:52.028112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.028112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.028112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:52.028112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.028112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:52.028112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:52.028417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.028754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.029148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:52.029148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.029148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:52.029148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.029359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:39:52.029359 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.029359 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:52.029359 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.029359 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.029359 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:52.029359 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.029359 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:52.030771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:52.031130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.031130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:52.031130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.031130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:52.031130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.031130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.031130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:52.031130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.031130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.032242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:52.032525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.032798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.032798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:52.032798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.032798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:52.035654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.036018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:52.036404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.036404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:52.036404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.036404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.036404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:52.036404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.036404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:52.038459 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:52.038811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:52.038927 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:52.039050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:52.041457 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:52.041457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.041457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:39:52.041457 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.041457 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:52.041457 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.041457 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.041457 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:52.041457 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.041457 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:52.041983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:52.042554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.042554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:52.042554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.042554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:52.042554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.042554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.042554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:52.042554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.042554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:52.043565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.043565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:52.043565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:52.043565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:52.043565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.043565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:52.043565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:52.043565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:52.043565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:52.043565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.043565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:52.043896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.044321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.044321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:52.044321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.044321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.046223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:52.046875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.047606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.047606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:52.047606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.047606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:52.116022 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:52.116746 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:52.116849 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:52.117044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:52.117569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:52.117874 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:52.118044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.118811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:39:52.118811 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.118811 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:52.118811 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.118811 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.118811 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:52.118811 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.118811 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.119584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:52.120141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:52.120687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:52.121201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:52.121840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:52.121840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:52.121840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:56.979469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:56.980471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:56.980471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:56.981106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:57.020897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:57.021199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.022579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:57.022579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:57.022958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:39:57.022958 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.022958 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:57.022958 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.022958 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.022958 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:57.022958 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.022958 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:57.023550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.024703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.026647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:57.027106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.027106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:57.027106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:57.027106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:57.027523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.029614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.030314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:57.030314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.030314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:57.030314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:57.030314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:57.030314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:39:57.030314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:57.030314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.030638 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:57.030638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.030908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:39:57.030908 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.030908 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:57.030908 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.030908 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.030908 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:57.030908 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.030908 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.033606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.034017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:57.034393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.034393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:57.034393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:39:57.034393 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.034393 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:57.034393 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.034393 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.034393 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:57.034393 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.034393 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:57.034852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:57.035784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.035784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:57.035784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.035784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:57.035784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.035784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.035784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:57.035784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.035784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:57.036332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.036793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:39:57.037390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.037772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:57.038253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.038253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:57.038253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.038253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.038253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:57.038253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.038253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:57.041684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.042484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:57.043129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.043129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:57.043129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.043129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.043129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:57.043129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.043129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:57.043129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:57.043129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:57.043129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:57.043618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:57.044987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:57.045257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.046299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:39:57.046299 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.046299 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:57.046299 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.046299 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.046299 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:57.046299 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.046299 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:39:57.048661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:57.049554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:57.050194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:57.052489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:39:57.053051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.053928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:39:57.116764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:39:57.117058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:39:57.117179 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:39:57.117420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:39:57.119611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:57.119873 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:39:57.120019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.120582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:39:57.120582 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.120582 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:39:57.120582 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.120582 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.120582 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:39:57.120582 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.120582 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.121891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:39:57.122500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.123014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:39:57.123592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:39:57.124398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:39:57.125068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:39:57.125068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:01.980242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:01.980898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:01.981083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:01.983014 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:02.022892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:02.023663 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:02.024829 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:02.025880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:02.027322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:02.027426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:02.027516 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:02.028430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:02.028892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:02.029380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.031449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:02.031449 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.031449 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:02.031449 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.031449 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.031449 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:02.031449 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.031449 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.032756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.033149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:02.033616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.033616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:02.033977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.034450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.034906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:02.034906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.034906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:02.034906 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:02.034906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.035324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:02.035324 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.035324 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:02.035324 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.035324 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.035324 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:02.035324 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.035324 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:02.036391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.037191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:02.037527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.037898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.037898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:02.037898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.037898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:02.037898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.038663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:02.039317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.039815 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_fdb_aging_settings failed after 0.058 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.041082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.041564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:02.041948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.041948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:02.042879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.043376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.043832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:02.043832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.043832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:02.043832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:02.043832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:02.043832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:02.044016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:02.044319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:02.045180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.045788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:40:02.045788 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.045788 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:02.045788 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.045788 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.045788 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:02.045788 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.045788 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:02.046583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:02.047125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.047125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:02.047125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.047125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:02.047125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.047125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.047125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:02.047125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.047125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:02.051254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.051254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:02.051254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:02.051254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:02.051254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.051254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:02.051254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:02.051254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:02.051254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:02.051254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.051254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:02.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.051936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.051936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:02.051936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.051936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.053798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:02.054249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.054726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.054726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:02.054726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.054726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:02.117530 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:02.118156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:02.118313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:02.118518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:02.119128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:02.119491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:02.119891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.120584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:02.120584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.120584 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:02.120584 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.120584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.120584 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:02.120584 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.120584 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.121787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:02.122428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.122942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:02.123138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:02.123865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:02.124645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:02.124645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:06.980857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:06.981794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:06.981794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:06.983416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:07.017609 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:07.017880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.018669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:07.018669 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.018669 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:07.018669 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.018669 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.018669 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:07.018669 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.018669 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.019543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.020256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:07.020965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:07.021805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.022802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:07.022802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.022802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.022802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:07.022802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.022802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:07.023593 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:07.024624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:07.024624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:07.025067 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:07.025459 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:07.025690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.026202 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:07.026677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:07.026882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:07.027089 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:07.028835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:07.028835 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.028835 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:07.028835 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.028835 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.028835 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:07.028835 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.028835 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.029883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.030254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:07.030588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.030588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:07.030954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.032274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.032748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:07.033225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.033225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:07.033225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.033225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:40:07.033225 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.033225 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:07.033225 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.033225 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.033225 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:07.033225 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.033225 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:07.034793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:07.035273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.035273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:07.035273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.035273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:07.035273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.035273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.035273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:07.035273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.035273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.036274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:07.036614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.036992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.036992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:07.036992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.036992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:07.038052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.038509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:07.038978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.038978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:07.038978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.038978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.038978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:07.038978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.038978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:07.043608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:07.044349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:07.044349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:07.044349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:07.045035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:07.045418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.046341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:40:07.046341 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.046341 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:07.046341 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.046341 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.046341 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:07.046341 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.046341 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:07.047039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:07.047734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.047734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:07.047734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.047734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:07.047734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.047734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.047734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:07.047734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.047734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:07.048095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.048095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:07.048095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:07.048095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:07.048095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.048095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:07.048095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:07.048095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:07.048095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:07.048095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.048095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:07.050082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:07.050960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:07.052009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.052571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:07.052571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:07.052571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.052571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:07.052571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.052571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:07.052571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.052571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.052571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:07.052571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.052571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:07.118325 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:07.118819 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:07.118900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:07.119070 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:07.119695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:07.120182 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:07.120512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.121619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:07.121619 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.121619 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:07.121619 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.121619 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.121619 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:07.121619 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.121619 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.122626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:07.123012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.123342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:07.123786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:07.124227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:07.124746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:07.124746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:11.981698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:11.982557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:11.982762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:11.984038 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:12.017517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:12.017967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.018923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:12.018923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.018923 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:12.018923 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.018923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.018923 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:12.018923 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.018923 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.019824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.020592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:12.021259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:12.021949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.022841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:12.022841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.022841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.022841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:12.022841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.022841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:12.024880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:12.025307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:12.025422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:12.026236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:12.026541 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:12.026994 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:12.027540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.028387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:12.028976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:12.029233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:12.029648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:12.029648 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.029648 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:12.029648 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.029648 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.029648 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:12.029648 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.029648 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.031048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.031472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:12.031816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.031816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:12.031816 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.032522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.033116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:12.033687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.033687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:12.033687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.033687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:40:12.033687 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.033687 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:12.033687 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.033687 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.033687 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:12.033687 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.033687 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:12.035771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:12.036170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.036170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:12.036170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.036170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:12.036170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.036170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.036170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:12.036170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.036170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.038096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:12.038495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.038856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.038856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:12.038856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.038856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:12.039108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.039592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:12.040225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.040225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:12.040225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.040225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.040225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:12.040225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.040225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:12.044063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:12.044395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:12.044520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:12.044679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:12.045031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:12.045220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.045756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:40:12.045756 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.045756 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:12.045756 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.045756 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.045756 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:12.045756 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.045756 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:12.047600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:12.048046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.048046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:12.048046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.048046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:12.048046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.048046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.048046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:12.048046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.048046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:12.049568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.049568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:12.049568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:12.049568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:12.049568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.049568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:12.049568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:12.049568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:12.049568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:12.049568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.049568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:12.049963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.050874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.050874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:12.050874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.050874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.051465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:12.052541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.053502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.053502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:12.053502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.053502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:12.119174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:12.119695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:12.119880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:12.120077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:12.120522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:12.120800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:12.120956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.121921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:12.121921 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.121921 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:12.121921 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.121921 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.121921 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:12.121921 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.121921 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.123079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:12.123885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.124382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:12.126640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:12.127231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:12.127775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:12.127775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:16.982438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:16.983120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:16.983269 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:16.984538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:17.022581 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:17.022982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.023913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:17.023913 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.023913 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:17.023913 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.023913 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.023913 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:17.023913 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.023913 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.025076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.025727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:17.026418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.026418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:17.026823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.028621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.029985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:17.029985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.029985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:17.029985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:17.029985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:17.029985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:17.029985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:17.029985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:17.031897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:17.031897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:17.031897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:17.031897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:17.031897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.031897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:17.031897 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.031897 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:17.031897 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.031897 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.031897 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.033023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.033763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:17.034372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:17.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.036454 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:17.036906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.037469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:17.037960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.038899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:17.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.040203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:17.045884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:17.046145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:17.046290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:17.046438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:17.047333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:17.047828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.048905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:40:17.048905 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.048905 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:17.048905 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.048905 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.048905 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:17.048905 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.048905 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:17.050416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:17.051193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.051193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:17.051193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.051193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:17.051193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.051193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.051193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:17.051193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.051193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:17.052987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.052987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:17.052987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:17.052987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:17.052987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.052987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:17.052987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:17.052987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:17.052987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:17.052987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.052987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:17.053428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.053792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.053792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:17.053792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.053792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.057440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:17.057913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.058443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.058443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:17.058443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.058443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:17.119818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:17.120934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:17.120934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:17.120934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:17.121492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:17.122045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:17.122408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.124122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:17.124122 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.124122 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:17.124122 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.124122 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.124122 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:17.124122 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.124122 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.126086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:17.126916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.127463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:17.128701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:17.129523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:17.130081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:17.130081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:21.983081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:21.984290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:21.984584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:21.985784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:22.017384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:22.017834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.018665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:22.018665 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.018665 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:22.018665 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.018665 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.018665 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:22.018665 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.018665 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.019380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.019925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:22.020407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:22.020844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.021389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:22.021389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.021389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.021389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:22.021389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.021389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:22.028132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:22.028552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:22.028719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:22.029840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:22.030587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:22.030986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:22.031272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:22.031773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:22.032185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:22.032566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.033628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:22.033628 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.033628 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:22.033628 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.033628 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.033628 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:22.033628 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.033628 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.034970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.035819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:22.036412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.036412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:22.036412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.036863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.037670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:22.038136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.038136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:22.038136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.038136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:40:22.038136 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.038136 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:22.038136 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.038136 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.038136 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:22.038136 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.038136 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:22.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:22.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:22.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:22.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:22.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.039375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.039855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:22.040186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.040552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.040552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:22.040552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.040552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:22.041178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.041648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:22.042128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.042128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:22.042128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.042128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.042128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:22.042128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.042128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:22.046361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:22.046725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:22.047260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:22.047260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:22.048074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:22.048856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.051398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:22.052447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.053270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.053872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:22.054494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:22.055633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.056436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:22.056436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.056436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:22.056436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.056436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.056436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:22.056436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.056436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:22.120665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:22.121173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:22.121367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:22.121611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:22.122203 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:22.122956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:22.123310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.125161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:22.125161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.125161 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:22.125161 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.125161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.125161 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:22.125161 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.125161 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.127247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:22.127897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.128895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:22.129236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:22.130001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:22.130875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:22.130875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:26.984326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:26.984933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:26.984992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:26.985937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:27.026672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:27.026925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.027807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:27.027807 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.027807 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:27.027807 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.027807 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.027807 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:27.027807 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.027807 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.028913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.029434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:27.029875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.029875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:27.029949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.030566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.032077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:27.032077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.032077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:27.032077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:27.032077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:27.032077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:27.032077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:27.033008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:27.033243 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:27.033443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:27.033751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:27.034479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:27.034587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.035270 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:27.035528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.036129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:40:27.036129 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.036129 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:27.036129 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.036129 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.036129 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:27.036129 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.036129 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:27.037894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:27.038536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.038536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:27.038536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.038536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:27.038536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.038536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.038536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:27.038536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.038536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.039776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:27.040268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.040703 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:27.041445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.042282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:27.042827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:27.043470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:27.043873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:27.044560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:27.045146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.045146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:27.045146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.045146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:27.045146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.045146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.045146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:27.045146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.045146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:27.046838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:27.047314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:27.047416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:27.047584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:27.048047 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:27.048339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.049003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:40:27.049003 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.049003 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:27.049003 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.049003 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.049003 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:27.049003 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.049003 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:27.049903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:27.050612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.050612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:27.050612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.050612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:27.050612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.050612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.050612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:27.050612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.050612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:27.050978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.050978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:27.050978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:27.050978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:27.050978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.050978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:27.050978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:27.050978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:27.050978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:27.050978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.050978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:27.051445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:27.051973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:27.052539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.053156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:27.053156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:27.053156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.053156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:27.053156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.053156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:27.053156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.053156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.053156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:27.053156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.053156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:27.122029 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:27.123081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:27.123226 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:27.123532 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:27.124422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:27.124912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:27.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.126681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:27.126681 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.126681 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:27.126681 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.126681 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.126681 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:27.126681 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.126681 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.128025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:27.128698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.129332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:27.129785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:27.130623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:27.131148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:27.131148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:31.985019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:31.985515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:31.985737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:31.987078 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:32.022946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:32.023469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.024607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:32.024607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.024607 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:32.024607 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.024607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.024607 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:32.024607 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.024607 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.025730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.026144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:32.026486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.026486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:32.026873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.027292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.027748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:32.027748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.027748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:32.032349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:32.032873 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:32.033621 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:32.034357 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:32.034702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:32.035543 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:32.035729 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:32.036515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:32.036936 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:32.037223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.038372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:40:32.038372 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.038372 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:32.038372 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.038372 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.038372 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:32.038372 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.038372 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.040416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.040823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:32.041163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.041163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:32.041163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:32.042620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.043118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.043645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:32.043645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.043645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:32.043645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.043928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:32.043928 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.043928 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:32.043928 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.043928 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.043928 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:32.043928 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.043928 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:32.044909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.045412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:32.045412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.045412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:32.045412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.045412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.045412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:32.045412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.045412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:32.045959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.046423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.046423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:32.046423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.046423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.046538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:32.046985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.047523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.047523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:32.047523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.047523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:32.047523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:32.047807 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:32.047939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:32.048370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:32.048840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:32.049092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.049712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:40:32.049712 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.049712 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:32.049712 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.049712 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.049712 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:32.049712 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.049712 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:32.050740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:32.051316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.051316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:32.051316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.051316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:32.051316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.051316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.051316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:32.051316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.051316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:32.051712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.051712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:32.051712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:32.051712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:32.051712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.051712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:32.051712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:32.051712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:32.051712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:32.051712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.051712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:32.052024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.052426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.052426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:32.052426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.052426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.053791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:32.054319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.054859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.054859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:32.054859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.054859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:32.123644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:32.124372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:32.124527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:32.124859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:32.125606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:32.126075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:32.126343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.127201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:32.127201 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.127201 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:32.127201 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.127201 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.127201 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:32.127201 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.127201 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.129466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:32.130052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.130512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:32.131599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:32.132269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:32.132845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:32.132845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:36.985812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:36.986739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:36.986739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:36.988053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:37.024882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:37.025333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.026300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:37.026300 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.026300 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:37.026300 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.026300 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.026300 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:37.026300 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.026300 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.027141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.028043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:37.028826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:37.029705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.030987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:37.030987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.030987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.030987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:37.030987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.030987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:37.034092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:37.034092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:37.034451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:37.034558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:37.035856 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:37.036330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:37.036713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:37.036998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:37.037422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:37.037760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.038633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:37.038633 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.038633 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:37.038633 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.038633 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.038633 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:37.038633 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.038633 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.039759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.040195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:37.040659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.040659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:37.040659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.041490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.042038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:37.042603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.042603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:37.042603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.042749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:40:37.042749 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.042749 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:37.042749 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.042749 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.042749 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:37.042749 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.042749 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:37.044652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.046416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:37.046826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.047259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.047259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:37.047259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.047259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:37.049989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.051155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:37.052326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.052326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:37.052326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.052326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.052326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:37.052326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.052326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:37.052326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:37.052326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:37.052326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:37.053038 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:37.053038 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:37.053038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.054431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:40:37.054431 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.054431 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:37.054431 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.054431 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.054431 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:37.054431 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.054431 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:37.057354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:37.057998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:37.058627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.059231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:37.059231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:37.059231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.059231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:37.059231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.059231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:37.059231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.059231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.059231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:37.059231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.059231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.059668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:37.060326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.060854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.060854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:37.060854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.060854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:37.124545 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:37.126380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:37.126380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:37.126380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:37.126380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:37.126726 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:37.126908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.128500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:37.128500 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.128500 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:37.128500 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.128500 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.128500 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:37.128500 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.128500 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.129533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:37.129972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:37.130466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:37.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:37.132143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:37.132143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:37.132143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:41.986680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:41.987509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:41.987509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:41.988771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:41.990979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=471,neutron:UPDATE=2 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:40:42.023348 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:42.023914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.025109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:42.025109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.025109 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:42.025109 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.025109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.025109 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:42.025109 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.025109 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.026199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.026583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:42.026983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.026983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:42.027324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.027776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.028552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:42.028552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.028552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:42.034574 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:42.034967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:42.035085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:42.035938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:42.036569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:42.036963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:42.037379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.038499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:42.039204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:42.039975 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:42.040517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:42.040517 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.040517 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:42.040517 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.040517 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.040517 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:42.040517 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.040517 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.042160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.042513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:42.042812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.042812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:42.042812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.044324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.044750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:42.045197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.045197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:42.045197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.045197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:40:42.045197 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.045197 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:42.045197 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.045197 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.045197 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:42.045197 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.045197 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:42.046435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:42.046864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.046864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:42.046864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.046864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:42.046864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.046864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.046864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:42.046864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.046864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.047719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:42.048058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.048455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.048455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:42.048455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.048455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:42.049357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.049757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:42.050231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.050231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:42.050231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.050231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.050231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:42.050231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.050231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:42.050905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:42.051583 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:42.051771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:42.051980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:42.052684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:42.053042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.053790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:40:42.053790 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.053790 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:42.053790 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.053790 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.053790 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:42.053790 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.053790 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:42.055126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:42.055513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.055513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:42.055513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.055513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:42.055513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.055513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.055513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:42.055513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.055513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:42.056607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.056607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:42.056607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:42.056607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:42.056607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.056607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:42.056607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:42.056607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:42.056607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:42.056607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.056607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:42.056922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.057296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.057296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:42.057296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.057296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.058355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:42.058767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.059292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.059292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:42.059292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.059292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:42.125491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:42.126130 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:42.126359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:42.126627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:42.127355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:42.127829 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:42.128108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.129141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:42.129141 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.129141 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:42.129141 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.129141 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.129141 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:42.129141 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.129141 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.130880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:42.131263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.131606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:42.132825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:42.133323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:42.133852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:42.133852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:46.987405 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:46.988170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:46.988263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:46.989899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:47.035607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:47.037941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:47.037941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:47.038735 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:47.039367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:47.039696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:47.040397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:47.041914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:47.043650 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:47.043908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.045098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:47.045098 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.045098 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:47.045098 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.045098 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.045098 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:47.045098 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.045098 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.048391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:47.052781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:47.053655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.054672 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:47.055725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.056560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.057379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.058057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:47.058458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:47.058832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.059317 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.059741 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:47.059741 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.059741 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:47.059741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.071 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.060685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.061062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:47.061471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.061471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:47.061471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:47.062876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.063387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.063910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:47.063910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.063910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:47.063910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:47.064317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:47.064659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.065330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:40:47.065330 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.065330 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:47.065330 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.065330 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.065330 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:47.065330 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.065330 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:47.066595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:47.067623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.067623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:47.067623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.067623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:47.067623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.067623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.067623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:47.067623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.067623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:47.068109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.068109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:47.068109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:47.068109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:47.068109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.068109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:47.068109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:47.068109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:47.068109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:47.068109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.068109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:47.068902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.069915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.069915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:47.069915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.069915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.070528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:47.071150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.071865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.071865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:47.071865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.071865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:47.126351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:47.127172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:47.127462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:47.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:47.128738 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:47.129318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:47.129763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.130931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:47.130931 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.130931 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:47.130931 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.130931 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.130931 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:47.130931 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.130931 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.132868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:47.133511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:47.134036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:47.134750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:47.135542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:47.135542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:47.135542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:51.988151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:51.989092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:51.989092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:51.989597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:52.028015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:52.028244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.028863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:40:52.028863 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.028863 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:52.028863 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.028863 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.028863 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:52.028863 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.028863 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.029741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.030167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:52.030659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.030659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:52.030910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.031303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.031659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:52.031659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.031659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:52.037732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:52.038432 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:52.038651 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:52.039510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:52.040223 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:52.040617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:52.040817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:52.041198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:52.041439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:52.041593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.042243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:40:52.042243 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.042243 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:52.042243 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.042243 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.042243 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:52.042243 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.042243 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.043318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.044011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:52.044727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:52.045485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.046395 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:52.047353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:52.047796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.047796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:52.047796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.047796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:52.047796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.047796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.047796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:52.047796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.047796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.048288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:52.048639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.049006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.049006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:52.049006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.049006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:52.049930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.050471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:52.050994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.050994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:52.050994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.050994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.050994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:52.050994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.050994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:52.054385 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:52.054993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:52.055110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:52.055283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:52.055665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:52.055897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.056485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:40:52.056485 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.056485 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:52.056485 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.056485 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.056485 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:52.056485 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.056485 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:52.057424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:52.057853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.057853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:52.057853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.057853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:52.057853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.057853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.057853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:52.057853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.057853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:52.058364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.058364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:52.058364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:52.058364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:52.058364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.058364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:52.058364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:52.058364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:52.058364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:52.058364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.058364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:52.058666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.059119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.059119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:52.059119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.059119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.059547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:52.059984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.060560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.060560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:52.060560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.060560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:52.127142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:52.128051 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:52.128299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:52.128584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:52.129489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:52.129956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:52.130280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.131445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:52.131445 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.131445 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:52.131445 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.131445 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.131445 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:52.131445 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.131445 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.133906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:52.134531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.135089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:52.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:52.138776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:52.139512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:52.139512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:56.988817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:56.989939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:56.989939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:56.990171 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:57.021904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:57.022301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.023290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:40:57.023290 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.023290 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:57.023290 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.023290 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.023290 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:57.023290 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.023290 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.024894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.025180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:57.025469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.025469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:57.026580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.026935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.027347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:57.027347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.027347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:57.038272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:57.038396 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:57.038505 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:57.039385 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:57.039745 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:57.040117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.040941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:40:57.040941 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.040941 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:57.040941 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.040941 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.040941 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:57.040941 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.040941 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.042433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.043028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:57.043595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.043595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:57.044253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.044944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.045779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:57.045779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.045779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:57.045779 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:57.045779 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:57.045779 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:57.045779 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:40:57.046267 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:57.046448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.047096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:40:57.047096 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.047096 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:57.047096 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.047096 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.047096 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:57.047096 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.047096 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:57.048519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:57.048992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.048992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:57.048992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.048992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:57.048992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.048992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.048992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:57.048992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.048992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.049857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:57.050258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.050639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.050639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:57.050639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.050639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:57.051554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.051975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:57.052503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.052503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:57.052503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.052503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.052503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:57.052503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.052503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:57.054803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:57.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:57.055556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:57.055776 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:57.056314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:57.056680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.057402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:40:57.057402 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.057402 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:57.057402 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.057402 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.057402 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:57.057402 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.057402 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:40:57.058709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:57.059866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:57.060696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:57.061359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:40:57.062262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.063145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:40:57.128282 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:40:57.129352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:40:57.129352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:40:57.129352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:40:57.130250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:57.130520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:40:57.130819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.131827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:40:57.131827 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.131827 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:40:57.131827 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.131827 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.131827 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:40:57.131827 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.131827 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.133443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:40:57.134147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.134963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:40:57.135576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:40:57.136653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:40:57.137504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:40:57.137504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:01.989491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:01.990548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:01.990658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:01.992118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:02.031773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:02.032528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.033867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:02.033867 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.033867 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:02.033867 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.033867 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.033867 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:02.033867 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.033867 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.035993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.036939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:02.038254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.038254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:02.038812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.039382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.040187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:02.040187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.040187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:02.040187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:02.040187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:02.040187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:02.040652 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:02.040917 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:02.041083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.041792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:41:02.041792 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.041792 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:02.041792 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.041792 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.041792 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:02.041792 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.041792 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.042826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.043305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:02.043803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.043803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:02.044203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.044661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.045115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:02.045115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.045115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:02.045115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:02.045321 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:02.045517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:02.045656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:02.046024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:02.046244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.046766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:02.046766 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.046766 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:02.046766 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.046766 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.046766 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:02.046766 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.046766 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:02.047644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:02.048566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.049488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.050354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:02.051093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.052130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:02.053259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.053259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:02.053259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.053259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.053259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:02.053259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.053259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:02.055442 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:02.055856 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:02.055984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:02.056131 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:02.056644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:02.057001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.057865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:41:02.057865 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.057865 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:02.057865 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.057865 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.057865 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:02.057865 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.057865 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:02.058823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:02.059204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.059204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:02.059204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.059204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:02.059204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.059204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.059204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:02.059204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.059204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:02.059879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.059879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:02.059879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:02.059879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:02.059879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.059879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:02.059879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:02.059879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:02.059879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:02.059879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.059879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:02.060174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.060604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.060604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:02.060604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.060604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.061189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:02.061813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.062429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.062429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:02.062429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.062429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:02.129092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:02.129404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:02.129471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:02.129647 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:02.130224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:02.130720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:02.131019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.131885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:02.131885 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.131885 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:02.131885 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.131885 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.131885 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:02.131885 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.131885 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:02.133417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.133799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:02.134852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:02.135757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:02.136797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:02.136797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:06.990371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:06.991315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:06.991315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:06.992468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:07.031129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:07.031620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.032722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:07.032722 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.032722 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:07.032722 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.032722 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.032722 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:07.032722 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.032722 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.033556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.034108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:07.034592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:07.035173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.035982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:07.035982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.035982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.035982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:07.035982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.035982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:07.039773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:07.040419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:07.040712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:07.042081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:07.042304 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:07.042568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.043523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:41:07.043523 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.043523 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:07.043523 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.043523 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.043523 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:07.043523 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.043523 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.044898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.045307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:07.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.045643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:07.046929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.047419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.047926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:07.047926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.047926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:07.047926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:07.048046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:07.048469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:07.048624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:07.049052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:07.049219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.051033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:07.051033 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.051033 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:07.051033 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.051033 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.051033 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:07.051033 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.051033 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:07.052351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:07.053133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.053865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.054401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:07.054957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.055740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:07.056834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.056834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:07.056834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.056834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.056834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:07.056834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.056834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:07.056834 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:07.056834 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:07.056834 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:07.057495 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:07.057495 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:07.057657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.058449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:41:07.058449 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.058449 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:07.058449 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.058449 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.058449 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:07.058449 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.058449 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:07.059371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:07.059970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.059970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:07.059970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.059970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:07.059970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.059970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.059970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:07.059970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.059970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:07.060571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.060571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:07.060571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:07.060571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:07.060571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.060571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:07.060571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:07.060571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:07.060571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:07.060571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.060571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:07.061032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.061622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.061622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:07.061622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.061622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.062020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:07.062666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.063337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.063337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:07.063337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.063337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:07.129719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:07.130470 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:07.130653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:07.130981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:07.131566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:07.131875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:07.132061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.132901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:07.132901 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.132901 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:07.132901 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.132901 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.132901 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:07.132901 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.132901 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.134689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:07.135420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:07.136642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:07.137627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:07.138772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:07.138772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:11.991173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:11.992385 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:11.992484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:11.994086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:12.040087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.041130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.041851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:12.042571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:12.046351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:12.047251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:12.047251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:12.047853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:12.049062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:12.049552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:12.049667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:12.049775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:12.051106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:12.051106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.051106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:12.051106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.051418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:41:12.051418 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.051418 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:12.051418 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.051418 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.051418 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:12.051418 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.051418 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.052627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.052995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:12.053263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.053263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:12.053607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:12.053607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.053607 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:12.053607 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.053607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.053607 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:12.053607 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.053607 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:12.054365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:12.054867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.054867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:12.054867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.054867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:12.054867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.054867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.054867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:12.054867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.054867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.055092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:12.055427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:12.057381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.057854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:12.058283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.059220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:12.060115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:12.060606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:12.060606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:12.060606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:12.061411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:12.061411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.061914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:41:12.061914 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.061914 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:12.061914 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.061914 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.061914 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:12.061914 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.061914 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:12.062689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:12.063120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.063120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:12.063120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.063120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:12.063120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.063120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.063120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:12.063120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.063120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:12.063690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.064031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.064031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:12.064031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.064031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.064156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:12.067247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.067750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.067750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:12.067750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.067750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:12.130692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:12.131369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:12.131533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:12.131787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:12.132464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:12.132977 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:12.133139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.134073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:12.134073 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.134073 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:12.134073 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.134073 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.134073 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:12.134073 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.134073 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.135643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:12.136276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.136958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:12.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:12.137742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:12.138271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:12.138271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:16.992119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:16.993966 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:16.993966 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:16.995289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:17.028910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:17.029416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.030668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:17.030668 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.030668 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:17.030668 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.030668 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.030668 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:17.030668 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.030668 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.032295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.032928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:17.033675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.033675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:17.033970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.034797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.035690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:17.035690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.035690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:17.045085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:17.046023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:17.046301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:17.047358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:17.047747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:17.048003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.048973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:17.048973 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.048973 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:17.048973 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.048973 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.048973 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:17.048973 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.048973 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.050937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.051586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:17.052313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.052313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:17.052313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:17.053154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.053994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.054969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:17.054969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.054969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:17.054969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:17.054969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:17.054969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:17.054969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:17.055403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.056004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:41:17.056004 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.056004 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:17.056004 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.056004 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.056004 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:17.056004 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.056004 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:17.057501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:17.057983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.057983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:17.057983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.057983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:17.057983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.057983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.057983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:17.057983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.057983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.059094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:17.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.059934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.059934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:17.059934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.059934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:17.061237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.061693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:17.062158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.062158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:17.062158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.062158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.062158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:17.062158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.062158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:17.062158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:17.062158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:17.062158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:17.062485 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:17.062764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:17.063058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.063584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:41:17.063584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.063584 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:17.063584 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.063584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.063584 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:17.063584 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.063584 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:17.064574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:17.065030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.065030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:17.065030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.065030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:17.065030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.065030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.065030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:17.065030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.065030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:17.065380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.065380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:17.065380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:17.065380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:17.065380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.065380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:17.065380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:17.065380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:17.065380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:17.065380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.065380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:17.065707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.066091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.066091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:17.066091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.066091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.066483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:17.066977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.067484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.067484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:17.067484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.067484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:17.131461 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:17.131946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:17.132021 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:17.132224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:17.133230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:17.133774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:17.134037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.135073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:17.135073 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.135073 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:17.135073 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.135073 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.135073 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:17.135073 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.135073 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.136190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:17.136850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:17.137416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:17.138112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:17.138967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:17.138967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:17.138967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:21.992855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:21.993864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:21.993864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:21.994416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:22.032882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:22.033139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.033740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:22.033740 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.033740 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:22.033740 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.033740 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.033740 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:22.033740 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.033740 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.034942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.035859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:22.036682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:22.037301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.037947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:22.037947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.037947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.037947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:22.037947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.037947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:22.046020 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:22.047083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:22.047326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:22.048681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:22.049058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:22.049349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.050323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:22.050323 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.050323 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:22.050323 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.050323 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.050323 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:22.050323 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.050323 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.052458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.053188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:22.053701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.053701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:22.053848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.054855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:22.054855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.054855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:22.054855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:22.054855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:22.054855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:22.055097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:22.055748 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:22.055995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.056610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:22.056610 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.056610 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:22.056610 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.056610 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.056610 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:22.056610 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.056610 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:22.057565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:22.058263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.058263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:22.058263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.058263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:22.058263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.058263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.058263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:22.058263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.058263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.058675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:22.059063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.059468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.059468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:22.059468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.059468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:22.059637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.060085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:22.060632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.060632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:22.060632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.060632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.060632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:22.060632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.060632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:22.061985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:22.062591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:22.062930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:22.063106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:22.063606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:22.064040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.065024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:41:22.065024 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.065024 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:22.065024 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.065024 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.065024 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:22.065024 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.065024 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:22.066034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:22.066487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.066487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:22.066487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.066487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:22.066487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.066487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.066487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:22.066487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.066487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:22.067368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.067368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:22.067368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:22.067368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:22.067368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.067368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:22.067368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:22.067368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:22.067368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:22.067368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.067368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:22.067694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.068087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.068087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:22.068087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.068087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.069128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:22.069641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.070115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.070115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:22.070115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.070115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:22.132279 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:22.133586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:22.133586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:22.133586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:22.133962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:22.134552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:22.135003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.135652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:22.135652 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.135652 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:22.135652 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.135652 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.135652 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:22.135652 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.135652 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:22.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:22.138151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:22.139289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:22.140359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:22.140359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:22.140359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:26.993763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:26.997775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:26.997775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:26.997775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:27.032026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:27.032575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.033321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:27.033321 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.033321 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:27.033321 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.033321 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.033321 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:27.033321 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.033321 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.034700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.034995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:27.035283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.035283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:27.036156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.036550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.036919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:27.036919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.036919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:27.046960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:27.047295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:27.047410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:27.048406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:27.048406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:27.048849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.049685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:27.049685 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.049685 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:27.049685 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.049685 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.049685 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:27.049685 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.049685 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.050307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.051088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:27.051879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:27.052839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.054021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:27.054021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.054021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.054021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:27.054021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.054021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:27.054884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:27.056820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:27.057053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:27.057339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:27.059366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:27.059639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.060820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:41:27.060820 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.060820 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:27.060820 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.060820 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.060820 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:27.060820 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.060820 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:27.062532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:27.063090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.063090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:27.063090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.063090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:27.063090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.063090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.063090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:27.063090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.063090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.063403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:27.063777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:27.064160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:27.065355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:27.065862 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:27.066361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:27.066736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:27.067282 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:27.067602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.068373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:41:27.068373 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.068373 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:27.068373 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.068373 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.068373 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:27.068373 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.068373 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:27.069365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:27.069875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.069875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:27.069875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.069875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:27.069875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.069875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.069875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:27.069875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.069875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:27.070355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.070355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:27.070355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:27.070355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:27.070355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.070355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:27.070355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:27.070355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:27.070355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:27.070355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.070355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:27.070755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.071155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.071155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:27.071155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.071155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.071649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:27.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.072937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.072937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:27.072937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.072937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:27.133608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:27.134330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:27.134512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:27.134933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:27.135398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:27.135774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:27.135979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.136864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:27.136864 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.136864 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:27.136864 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.136864 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.136864 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:27.136864 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.136864 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.138774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:27.139192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:27.140958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:27.141914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:27.142895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:27.142895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:31.995326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:31.996172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:31.996172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:31.997720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:32.034249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:32.034476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.035106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:32.035106 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.035106 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:32.035106 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.035106 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.035106 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:32.035106 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.035106 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.036244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.036877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:32.037502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:32.038240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.039035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:32.039035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.039035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.039035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:32.039035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.039035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:32.047506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:32.048438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:32.048652 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:32.049687 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:32.049981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:32.050130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.050903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:32.050903 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.050903 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:32.050903 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.050903 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.050903 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:32.050903 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.050903 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.053023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.053736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:32.054482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.054482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:32.054921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.055855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.056957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:32.056957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.056957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:32.056957 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:32.056957 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:32.057259 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:32.057259 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:32.057788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:32.058071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.058835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:41:32.058835 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.058835 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:32.058835 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.058835 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.058835 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:32.058835 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.058835 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:32.060336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:32.060786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.060786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:32.060786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.060786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:32.060786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.060786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.060786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:32.060786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.060786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.061528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:32.061928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.062352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.062352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:32.062352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.062352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:32.063557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.064005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:32.064536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.064536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:32.064536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.064536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.064536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:32.064536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.064536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:32.065467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:32.066064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:32.066202 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:32.066404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:32.067047 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:32.067475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.068310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:41:32.068310 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.068310 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:32.068310 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.068310 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.068310 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:32.068310 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.068310 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:32.069231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:32.069654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.069654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:32.069654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.069654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:32.069654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.069654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.069654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:32.069654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.069654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:32.070184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.070184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:32.070184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:32.070184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:32.070184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.070184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:32.070184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:32.070184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:32.070184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:32.070184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.070184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:32.070563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.070931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.070931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:32.070931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.070931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.071391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:32.071819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.072374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.072374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:32.072374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.072374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:32.134262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:32.135038 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:32.135280 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:32.135629 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:32.136349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:32.136778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:32.137011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.137993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:32.137993 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.137993 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:32.137993 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.137993 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.137993 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:32.137993 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.137993 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.140229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:32.140945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.141623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:32.143387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:32.144293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:32.145154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:32.145154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:36.996369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:36.997366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:36.997445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:36.998799 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:37.033481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:37.033705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.034390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:37.034390 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.034390 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:37.034390 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.034390 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.034390 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:37.034390 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.034390 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.035120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.036078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:37.036905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:37.037909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.038676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:37.038676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.038676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.038676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:37.038676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.038676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:37.048512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:37.048939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:37.049128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:37.050202 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:37.050717 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:37.051388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.051890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:37.051890 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.051890 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:37.051890 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.051890 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.051890 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:37.051890 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.051890 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.053646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.054139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:37.054812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.054812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:37.055580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.056094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.056723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:37.056723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.056723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:37.056723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:37.057191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:37.057344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:37.057483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:37.058261 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:37.058261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.058853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:41:37.058853 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.058853 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:37.058853 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.058853 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.058853 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:37.058853 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.058853 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:37.060134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:37.060666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.060666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:37.060666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.060666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:37.060666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.060666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.060666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:37.060666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.060666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.061433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:37.061911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.062383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.062383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:37.062383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.062383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:37.062739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.063348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:37.063903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.063903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:37.063903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.063903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.063903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:37.063903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.063903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:37.066003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:37.066787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:37.066984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:37.067201 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:37.067894 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:37.068421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.069325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:41:37.069325 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.069325 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:37.069325 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.069325 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.069325 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:37.069325 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.069325 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:37.070695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:37.070996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.070996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:37.070996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.070996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:37.070996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.070996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.070996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:37.070996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.070996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:37.072559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.072559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:37.072559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:37.072559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:37.072559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.072559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:37.072559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:37.072559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:37.072559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:37.072559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.072559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:37.073587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.074374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.074374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:37.074374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.074374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.074908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:37.075549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.076142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.076142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:37.076142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.076142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:37.135006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:37.135776 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:37.135962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:37.136226 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:37.137115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:37.137634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:37.137878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.138846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:37.138846 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.138846 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:37.138846 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.138846 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.138846 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:37.138846 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.138846 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.140435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:37.141274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.141859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:37.142318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:37.143043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:37.143531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:37.143531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:41.997173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:41.998486 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:41.998486 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:41.998782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:42.004488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=241 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:41:42.038150 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:42.038757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.039627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:42.039627 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.039627 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:42.039627 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.039627 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.039627 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:42.039627 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.039627 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.040677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.041483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:42.042183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:42.043042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.044030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:42.044030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.044030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.044030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:42.044030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.044030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:42.048960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:42.049938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:42.050224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:42.052166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:42.052797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:42.053178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.054688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:42.054688 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.054688 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:42.054688 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.054688 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.054688 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:42.054688 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.054688 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.056561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.057332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:42.058579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.058579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:42.058960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.061347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.063088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:42.063088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.063088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:42.063088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:42.063088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:42.063088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:42.063088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:42.063088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:42.063852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.063852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:41:42.063852 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.063852 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:42.063852 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.063852 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.063852 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:42.063852 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.063852 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:42.065041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:42.065423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.065423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:42.065423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.065423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:42.065423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.065423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.065423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:42.065423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.065423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.066484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:42.066738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.066988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.066988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:42.066988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.066988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:42.066988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:42.068311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.068661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:42.069036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.069036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:42.069036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.069036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.069036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:42.069036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.069036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:42.069036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:42.069036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:42.069036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:42.069733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:42.069990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.070576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:42.070576 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.070576 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:42.070576 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.070576 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.070576 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:42.070576 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.070576 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:42.071958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.072386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:42.072386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.072386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:42.072386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.072386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.072386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:42.072386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.072386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:42.073542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.073988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.074493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:42.074493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.074493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.075233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:42.075697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.076146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.076146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:42.076146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.076146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:42.135896 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:42.136912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:42.136912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:42.136912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:42.138488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:42.138960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:42.139063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.139874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:42.139874 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.139874 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:42.139874 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.139874 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.139874 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:42.139874 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.139874 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.140871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:42.141535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:42.142036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:42.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:42.143613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:42.143613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:42.143613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:46.997764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:46.998553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:46.998692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:46.999659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:47.041231 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:47.041717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.042815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:47.042815 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.042815 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:47.042815 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.042815 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.042815 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:47.042815 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.042815 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.044071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.044751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:47.045561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:47.046627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.049452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:47.049452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.049452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.049452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:47.049452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.049452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:47.050172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:47.050736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:47.050940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:47.052422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:47.052809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:47.053056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.054028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:47.054028 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.054028 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:47.054028 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.054028 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.054028 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:47.054028 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.054028 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.055714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.056764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:47.057844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:47.058712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.059323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:47.059323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.059323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.059323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:47.059323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.059323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:47.060030 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:47.060601 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:47.060698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:47.060825 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:47.061521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:47.061880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.062671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:41:47.062671 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.062671 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:47.062671 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.062671 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.062671 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:47.062671 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.062671 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:47.063436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:47.063968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.063968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:47.063968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.063968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:47.063968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.063968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.063968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:47.063968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.063968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.064235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:47.064766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:47.065328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.065894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.066690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:47.068869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:47.068869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:47.069129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:47.069387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:47.069943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:47.070183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.070756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:47.070756 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.070756 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:47.070756 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.070756 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.070756 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:47.070756 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.070756 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:47.072377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.073003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:47.073003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.073003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:47.073003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.073003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.073003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:47.073003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.073003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:47.074047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.074422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.074662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:47.074662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.074662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.076722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:47.077056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:47.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:47.136818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:47.137421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:47.137577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:47.137859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:47.138546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:47.139086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:47.139357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.140324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:47.140324 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.140324 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:47.140324 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.140324 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.140324 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:47.140324 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.140324 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.141316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:47.141691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.141991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:47.142348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:47.142785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:47.143234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:47.143234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:51.998466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:51.999481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:51.999647 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:52.000814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:52.050756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:52.051325 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:52.051484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:52.052410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:52.052835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:52.056283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:52.058334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:52.058715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.058715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:52.058715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.058715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:52.058715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.058715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.058715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:52.058715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.058715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:52.058715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:52.059014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.061012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:52.062636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:52.063073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:52.063415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.063415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:52.063415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.063415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:52.063415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.063415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.063415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:52.063415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.063415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:52.063415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:52.063646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.063974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.064721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:52.064721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.064721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:52.064721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:52.064721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:52.064721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:52.064721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:52.064937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.065780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:41:52.065780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.065780 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:52.065780 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.065780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.065780 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:52.065780 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.065780 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:52.067183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:52.067536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.067536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:52.067536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.067536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:52.067536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.067536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.067536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:52.067536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.067536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.068633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:52.068878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.069122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.069122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:52.069122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.069122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:52.069371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:52.071698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.072036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:52.072844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.072844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:52.072844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.072844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.072844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:52.072844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.072844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:52.072844 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:52.073013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:52.073184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:52.074048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:52.074525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.075963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:52.075963 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.075963 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:52.075963 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.075963 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.075963 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:52.075963 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.075963 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:52.076466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.076947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:52.076947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.076947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:52.076947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.076947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.076947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:52.076947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.076947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:52.077346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.077758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.078161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:52.078161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.078161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.078857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:52.079347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.080036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.080036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:52.080036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.080036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:52.137493 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:52.138305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:52.138305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:52.138305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:52.138941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:52.139264 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:52.139428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:52.141934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.142295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:52.142534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:52.142805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:52.143135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:52.143135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:52.143135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:52.143135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:52.143135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:52.143135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:52.143135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:52.143135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:52.143135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:56.999291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:57.000161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:57.000161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:57.000386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:57.041279 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:57.041762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.042902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:41:57.042902 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.042902 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:57.042902 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.042902 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.042902 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:57.042902 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.042902 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.043748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.045310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:57.046072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:57.046639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.047495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:57.047495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.047495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.047495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:57.047495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.047495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:57.051705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:57.052377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:57.052647 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:57.053652 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:57.053859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:57.054065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.054721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:41:57.054721 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.054721 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:57.054721 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.054721 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.054721 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:57.054721 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.054721 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.055899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.057070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:57.057908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:57.058937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.060756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:57.060756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.060756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.060756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:57.060756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.060756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:57.061556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:57.062091 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:57.062224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:57.062362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:41:57.062893 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:57.063147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.064371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:41:57.064371 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.064371 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:57.064371 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.064371 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.064371 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:57.064371 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.064371 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:57.065354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:57.066514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.068613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:57.069706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.070854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:57.072176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.072176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:57.072176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.072176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.072176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:57.072176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.072176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:57.072176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:57.073112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:57.073112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:57.073112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:57.074431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:57.075292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.077077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:41:57.077077 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.077077 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:57.077077 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.077077 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.077077 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:57.077077 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.077077 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:41:57.078434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.079084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:41:57.079084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.079084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:41:57.079084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.079084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.079084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:41:57.079084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.079084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:57.079792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.080301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.080713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:57.080713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.080713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.081822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:57.082632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.083275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.083275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:57.083275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.083275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:41:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:41:57.138946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:41:57.139173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:41:57.139481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:41:57.140302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:57.140790 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:41:57.141044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.142100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:41:57.142100 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.142100 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:41:57.142100 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.142100 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.142100 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:41:57.142100 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.142100 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.143679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.144341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:41:57.144990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:41:57.145924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:41:57.146794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:41:57.146794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:41:57.146794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:41:57.146794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:41:57.146794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:41:57.146794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:01.999967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:02.001260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:02.001260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:02.001260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:02.036445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:02.036815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.037455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:02.037455 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.037455 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:02.037455 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.037455 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.037455 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:02.037455 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.037455 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.038535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.038934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:02.039306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.039306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:02.039690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.040147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.040750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:02.040750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.040750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:02.052380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:02.052973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:02.053325 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:02.055761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:02.056234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:02.056387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.057426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:02.057426 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.057426 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:02.057426 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.057426 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.057426 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:02.057426 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.057426 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.058608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.059055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:02.059489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.059489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:02.059798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.060301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.060871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:02.060871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.060871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:02.061944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:02.062309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:02.062421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:02.062549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:02.066769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:02.067012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.067668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:42:02.067668 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.067668 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:02.067668 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.067668 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.067668 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:02.067668 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.067668 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:02.068490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:02.068938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.068938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:02.068938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.068938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:02.068938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.068938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.068938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:02.068938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.068938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.069524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:02.069889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.070308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.070308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:02.070308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.070308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:02.070683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.071113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:02.071610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.071610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:02.071610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.071610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.071610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:02.071610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.071610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:02.072598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:02.073608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:02.073778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:02.073966 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:02.077555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:02.077675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.078445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:42:02.078445 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.078445 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:02.078445 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.078445 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.078445 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:02.078445 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.078445 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:02.079954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:02.080389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.080389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:02.080389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.080389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:02.080389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.080389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.080389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:02.080389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.080389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:02.081989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.081989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:02.081989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:02.081989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:02.081989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.081989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:02.081989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:02.081989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:02.081989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:02.081989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.081989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:02.082373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.082710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.082710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:02.082710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.082710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.083222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:02.083653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.084192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.084192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:02.084192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.084192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:02.139185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:02.139817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:02.139924 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:02.140175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:02.140817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:02.140973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:02.141069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.141698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:42:02.141698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.141698 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:02.141698 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.141698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.141698 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:02.141698 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.141698 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.143061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:02.143728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:02.144310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:02.145074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:02.145885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:02.145885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:02.145885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:07.000693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:07.001497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:07.001634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:07.002607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:07.036352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:07.036662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.037507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:07.037507 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.037507 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:07.037507 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.037507 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.037507 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:07.037507 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.037507 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.038416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.039446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:07.040194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:07.041400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.042358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:07.042358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.042358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.042358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:07.042358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.042358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:07.053125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:07.053857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:07.053970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:07.055146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:07.055451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:07.055615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.056558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:07.056558 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.056558 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:07.056558 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.056558 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.056558 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:07.056558 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.056558 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.057735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.058418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:07.059118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:07.059897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.060931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:07.060931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.060931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.060931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:07.060931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.060931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:07.062349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:07.063053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:07.063317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:07.063537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:07.063977 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:07.064195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.064952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:42:07.064952 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.064952 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:07.064952 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.064952 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.064952 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:07.064952 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.064952 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:07.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:07.066559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.066559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:07.066559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.066559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:07.066559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.066559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.066559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:07.066559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.066559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.066991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:07.067383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.067735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.067735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:07.067735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.067735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:07.068064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.068557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:07.069012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.069012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:07.069012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.069012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.069012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:07.069012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.069012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:07.073437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:07.073903 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:07.074103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:07.074402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:07.074953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:07.075350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.076143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Traceback (most recent call last):
Apr 21 13:42:07.076143 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.076143 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:07.076143 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.076143 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.076143 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:07.076143 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.076143 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:07.077533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:07.077953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.077953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:07.077953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.077953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:07.077953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.077953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.077953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:07.077953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.077953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:07.079225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.079225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:07.079225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:07.079225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:07.079225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.079225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:07.079225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:07.079225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:07.079225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:07.079225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.079225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:07.079838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.080511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.080511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:07.080511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.080511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.081444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:07.082666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.083605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.083605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:07.083605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.083605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:07.139962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:07.140521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:07.140617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:07.140840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:07.141680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:07.142244 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:07.142590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.143588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:42:07.143588 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.143588 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:07.143588 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.143588 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.143588 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:07.143588 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.143588 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:07.149400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:07.149767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:07.150413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:07.150911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:07.150911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:07.150911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:12.003700 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:12.004318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:12.004401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:12.005791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:12.041655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:12.041917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.042629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:12.042629 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.042629 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:12.042629 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.042629 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.042629 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:12.042629 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.042629 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.043752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.044173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.044635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:12.044823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.045478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.046245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:12.046245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.046245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:12.053755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:12.054553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:12.054673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:12.055870 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:12.056083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:12.056225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.056828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:12.056828 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.056828 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:12.056828 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.056828 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.056828 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:12.056828 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.056828 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.058661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.059451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:12.060005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:12.060782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.061480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:12.061480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.061480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.061480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:12.061480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.061480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:12.062958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:12.063389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:12.063597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:12.063861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:12.064462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:12.064664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.065343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:42:12.065343 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.065343 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:12.065343 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.065343 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.065343 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:12.065343 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.065343 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:12.066693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:12.067079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.067079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:12.067079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.067079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:12.067079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.067079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.067079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:12.067079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.067079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.068445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:12.068811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.069152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.069152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:12.069152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.069152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:12.069591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.070006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:12.070403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.070403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:12.070403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.070403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.070403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:12.070403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.070403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:12.073818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:12.082114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:12.082294 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:12.082357 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:12.082966 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:12.083394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.084171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Traceback (most recent call last):
Apr 21 13:42:12.084171 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.084171 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:12.084171 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.084171 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.084171 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:12.084171 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.084171 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:12.084872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:12.085510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.085510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:12.085510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.085510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:12.085510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.085510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.085510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:12.085510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.085510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:12.085822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.085822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:12.085822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:12.085822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:12.085822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.085822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:12.085822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:12.085822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:12.085822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:12.085822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.085822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:12.086280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.086805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.086805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:12.086805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.086805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.088352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:12.089174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.089980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.089980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:12.089980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.089980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:12.140639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:12.141111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:12.141268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:12.141388 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:12.142044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:12.142500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:12.142739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.143556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:42:12.143556 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.143556 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:12.143556 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.143556 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.143556 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:12.143556 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.143556 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.144337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.144857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:12.145387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:12.146003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:12.146747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:12.146747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:12.146747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:12.146747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:12.146747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:12.146747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:17.004722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:17.005752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:17.005752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:17.006583 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:17.047076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:17.047668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.048790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:17.048790 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.048790 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:17.048790 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.048790 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.048790 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:17.048790 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.048790 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.049706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.050377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:17.051009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.051009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:17.051159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.051962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.052946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:17.052946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.052946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:17.054345 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:17.055256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:17.055256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:17.055762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:17.056109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:17.056362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:17.059320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:17.059788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.060183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.060603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:17.061121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:17.061121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.061121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:17.061121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.061121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:17.061121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.061121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.061121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:17.061121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.061121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:17.063497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:17.064045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:17.064273 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:17.064483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:17.065139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:17.065492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.066282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:42:17.066282 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.066282 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:17.066282 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.066282 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.066282 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:17.066282 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.066282 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:17.067670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:17.068118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.068118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:17.068118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.068118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:17.068118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.068118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.068118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:17.068118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.068118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.069227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:17.069648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.070044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.070044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:17.070044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.070044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:17.071454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.072179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:17.073081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.073081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:17.073081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.073081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.073081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:17.073081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.073081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:17.082080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:17.083135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:17.083359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:17.083747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:17.084480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:17.084945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.086134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:42:17.086134 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.086134 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:17.086134 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.086134 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.086134 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:17.086134 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.086134 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:17.087549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:17.088257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.088257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:17.088257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.088257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:17.088257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.088257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.088257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:17.088257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.088257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:17.090742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.090742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:17.090742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:17.090742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:17.090742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.090742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:17.090742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:17.090742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:17.090742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:17.090742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.090742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:17.091618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.092415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.092415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:17.092415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.092415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.093073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:17.094092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.095019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.095019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:17.095019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.095019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:17.141460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:17.142014 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:17.142118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:17.142281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:17.142945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:17.143236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:17.143477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.144137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:42:17.144137 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.144137 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:17.144137 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.144137 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.144137 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:17.144137 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.144137 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.145690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:17.146489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:17.147277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:17.148391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:17.148391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:17.148391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:17.148391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:17.148391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:17.148391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:22.005471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:22.006167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:22.006167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:22.006543 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:22.050429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:22.050663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.051520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:22.051520 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.051520 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:22.051520 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.051520 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.051520 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:22.051520 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.051520 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.053688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.054039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:22.054369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.054369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:22.055808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.057793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.059272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:22.059272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.059272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:22.059272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:22.059272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:22.059272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:22.059272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:22.059272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:22.059538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.059975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:22.059975 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.059975 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:22.059975 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.059975 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.059975 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:22.059975 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.059975 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.063368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.063818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:22.064280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.064280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:22.065138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.065583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:22.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:22.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:22.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:22.066055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:22.066234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:22.069251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.069747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:22.070116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:22.070367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.070698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:22.070698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.070698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.070698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:22.070698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.070698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:22.071130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.071563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:22.071961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.071961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:22.071961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.071961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.071961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:22.071961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.071961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:22.083101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:22.084157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:22.084157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:22.084157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:22.084731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:22.085134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.088128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:22.088597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.088978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.089299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:22.090879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:22.091337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.091864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:22.091864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.091864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:22.091864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.091864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.091864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:22.091864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.091864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:22.142110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:22.144429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:22.144573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:22.144757 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:22.145666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:22.146052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:22.146299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.147234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:42:22.147234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.147234 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:22.147234 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.147234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.147234 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:22.147234 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.147234 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.148358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.148911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:22.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:22.150371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:22.151352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:22.151352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:22.151352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:22.151352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:22.151352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:22.151352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:27.005970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:27.007092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:27.007092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:27.008109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:27.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:27.054406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.054716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:27.054716 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.054716 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:27.054716 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.054716 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.054716 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:27.054716 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.054716 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.055789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.057731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:27.058519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:27.059025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.059652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:27.059652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.059652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.059652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:27.059652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.059652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:27.059652 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:27.059652 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:27.059652 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:27.066393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:27.067757 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:27.068514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:27.068914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:27.069078 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:27.069237 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:27.070304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.070304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:27.070304 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.070304 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:27.070304 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.070304 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.070304 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:27.070304 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.070304 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.072560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.072915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:27.073439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.073439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:27.073735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.074984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.075569 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:27.075877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:27.076246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.076246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:27.076246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.076246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:27.076246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.076246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.076246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:27.076246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.076246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.080348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:27.080798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:27.081241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.081629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.082205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:27.083839 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:27.084451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:27.084576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:27.084729 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:27.087323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:27.087323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.087561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:42:27.087561 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.087561 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:27.087561 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.087561 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.087561 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:27.087561 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.087561 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:27.088364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:27.088709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.088709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:27.088709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.088709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:27.088709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.088709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.088709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:27.088709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.088709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:27.089067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.089067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:27.089067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:27.089067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:27.089067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.089067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:27.089067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:27.089067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:27.089067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:27.089067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.089067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:27.089335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.089718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.089718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:27.089718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.089718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.090926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:27.091525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.092138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.092138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:27.092138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.092138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:27.144551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:27.145249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:27.145403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:27.145729 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:27.147540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:27.147540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:27.147540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.148486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:42:27.148486 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.148486 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:27.148486 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.148486 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.148486 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:27.148486 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.148486 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.150513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.150835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:27.151181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.151181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:27.152102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:27.152968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:27.153828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:27.153828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:27.153828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:32.006865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:32.007540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:32.007641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:32.008517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:32.051121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:32.051942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.054038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:32.054038 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.054038 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:32.054038 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.054038 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.054038 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:32.054038 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.054038 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.055189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.055668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:32.056677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:32.057314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.057775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:32.057775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.057775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.057775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:32.057775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.057775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:32.058446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:32.058446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:32.058579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:32.059418 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:32.059506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:32.059662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.060302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:32.060302 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.060302 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:32.060302 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.060302 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.060302 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:32.060302 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.060302 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.061227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.061870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:32.062385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:32.062773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.063268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:32.063268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.063268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.063268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:32.063268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.063268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:32.068197 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:32.068648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:32.068749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:32.068867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:32.069547 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:32.069990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.071061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:42:32.071061 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.071061 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:32.071061 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.071061 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.071061 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:32.071061 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.071061 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:32.071747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:32.072437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.074068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.074952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:32.075539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.076312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:32.077141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.077141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:32.077141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.077141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.077141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:32.077141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.077141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:32.084481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:32.084983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:32.085199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:32.086465 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:32.086866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:32.088145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.088145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:42:32.088145 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.088145 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:32.088145 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.088145 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.088145 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:32.088145 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.088145 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:32.090323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:32.090882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.090882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:32.090882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.090882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:32.090882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.090882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.090882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:32.090882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.090882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:32.091569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.091569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:32.091569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:32.091569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:32.091569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.091569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:32.091569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:32.091569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:32.091569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:32.091569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.091569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:32.091950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.092405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.092405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:32.092405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.092405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.093374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:32.093868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.094456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.094456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:32.094456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.094456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:32.146303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:32.146303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:32.146303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:32.146303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:32.146815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:32.147050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:32.147249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.147939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:42:32.147939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.147939 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:32.147939 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.147939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.147939 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:32.147939 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.147939 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.149026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.149511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:32.149919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.149919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:32.150236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:32.150779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:32.151977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:32.151977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:32.151977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:37.007676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:37.008793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:37.008793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:37.008976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:37.040072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:37.040398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.041253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:37.041253 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.041253 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:37.041253 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.041253 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.041253 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:37.041253 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.041253 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.042325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.043503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:37.044368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:37.045336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.046744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:37.046744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.046744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.046744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:37.046744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.046744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:37.058784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:37.059491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:37.059687 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:37.061795 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:37.062119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:37.062314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.063102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:37.063102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.063102 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:37.063102 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.063102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.063102 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:37.063102 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.063102 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.064691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.065059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:37.065548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.065548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:37.066612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.067398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.068035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:37.068035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.068035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:37.068807 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:37.069181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:37.069321 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:37.069469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:37.070023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:37.070253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.070811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:42:37.070811 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.070811 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:37.070811 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.070811 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.070811 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:37.070811 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.070811 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:37.071732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:37.072676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.072676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:37.072676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.072676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:37.072676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.072676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.072676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:37.072676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.072676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.075473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:37.075923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:37.076292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.076966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.077793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:37.085092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:37.085646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:37.085784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:37.085953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:37.086523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:37.086838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.087584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:42:37.087584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.087584 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:37.087584 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.087584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.087584 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:37.087584 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.087584 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:37.089123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.089534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:37.089534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.089534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:37.089534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.089534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.089534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:37.089534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.089534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:37.091588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.092242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.092824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:37.092824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.092824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.094269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:37.096515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.097561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.097561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:37.097561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.097561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:37.146005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:37.147827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:37.147930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:37.148115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:37.148673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:37.148988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:37.149393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.150580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:42:37.150580 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.150580 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:37.150580 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.150580 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.150580 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:37.150580 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.150580 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.151400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.152118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:37.152630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:37.153269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:37.153982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:37.153982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:37.153982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:37.153982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:37.153982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:37.153982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:42.008611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:42.009408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:42.009408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:42.010258 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:42.011660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=239 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:42:42.047363 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:42.047363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.048255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:42.048255 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.048255 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:42.048255 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.048255 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.048255 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:42.048255 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.048255 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.048931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.049410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:42.049789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:42.050362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.050905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:42.050905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.050905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.050905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:42.050905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.050905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:42.059462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:42.059818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:42.059933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:42.060913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:42.061190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:42.061373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.062131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:42.062131 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.062131 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:42.062131 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.062131 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.062131 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:42.062131 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.062131 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.063716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.064084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:42.064484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.064484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:42.065554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.066039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.067031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:42.067031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.067031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:42.069233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:42.069713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:42.069833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:42.069984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:42.070492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:42.070828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.071656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:42:42.071656 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.071656 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:42.071656 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.071656 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.071656 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:42.071656 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.071656 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:42.072522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:42.073253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.073253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:42.073253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.073253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:42.073253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.073253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.073253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:42.073253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.073253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.073677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:42.074288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:42.074910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.075610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.076548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:42.085656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:42.086331 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:42.086418 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:42.086571 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:42.087479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:42.087912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.088789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:42:42.088789 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.088789 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:42.088789 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.088789 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.088789 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:42.088789 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.088789 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:42.089866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.090530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:42.090530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.090530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:42.090530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.090530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.090530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:42.090530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.090530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:42.090896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.092182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:42.092182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.092182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.093601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:42.094259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.094823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.094823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:42.094823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.094823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:42.148097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:42.148522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:42.148522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:42.148661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:42.149239 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:42.149522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:42.149630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.150186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:42:42.150186 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.150186 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:42.150186 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.150186 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.150186 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:42.150186 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.150186 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.151048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.151716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:42.152554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:42.153199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:42.154088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:42.154088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:42.154088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:42.154088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:42.154088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:42.154088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:47.009514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:47.010646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:47.010646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:47.012007 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:47.059831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:47.061370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:47.061466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.061882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:47.062057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:47.062998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:47.062998 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.062998 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:47.062998 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.062998 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.062998 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:47.062998 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.062998 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.064427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.064880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:47.065305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.065305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:47.066089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.066685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.067369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:47.067369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.067369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:47.067813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:47.068186 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:47.068411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.069129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:47.069129 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.069129 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:47.069129 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.069129 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.069129 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:47.069129 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.069129 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.070628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.071184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:47.071718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.071718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:47.072248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.073660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.074599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:47.074599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.074599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:47.074599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:47.074599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:47.074599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:47.074599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:47.074599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:47.078128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.078987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:47.078987 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.078987 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:47.078987 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.078987 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.078987 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:47.078987 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.078987 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:47.079727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:47.080235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.080235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:47.080235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.080235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:47.080235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.080235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.080235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:47.080235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.080235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.080494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:47.080781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.081080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.081080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:47.081080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.081080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:47.081434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.081821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:47.082285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.082285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:47.082285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.082285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.082285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:47.082285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.082285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:47.086346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:47.087222 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:47.087379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:47.087565 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:47.088363 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:47.088662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.089463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:42:47.089463 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.089463 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:47.089463 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.089463 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.089463 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:47.089463 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.089463 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:47.090590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:47.091122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.091122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:47.091122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.091122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:47.091122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.091122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.091122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:47.091122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.091122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:47.095225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.095225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:47.095225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:47.095225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:47.095225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.095225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:47.095225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:47.095225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:47.095225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:47.095225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.095225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:47.095757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.096324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.096324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:47.096324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.096324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.096992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:47.098454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.098981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.098981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:47.098981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.098981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:47.149572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:47.150539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:47.150699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:47.150905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:47.151742 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:47.152356 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:47.152574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.153653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:42:47.153653 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.153653 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:47.153653 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.153653 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.153653 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:47.153653 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.153653 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.154698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.155503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:47.155933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:47.156439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:47.157936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:47.157936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:47.157936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:47.157936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:47.157936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:47.157936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:52.010286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:52.011115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:52.011366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:52.012763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:52.056893 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:52.057278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.058128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:52.058128 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.058128 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:52.058128 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.058128 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.058128 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:52.058128 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.058128 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.060371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.061043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:52.061403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.061403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:52.063459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.063860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.064365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:52.064365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.064365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:52.064365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:52.064365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:52.064365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:52.065233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:52.065850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:52.065850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.067169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:42:52.067169 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.067169 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:52.067169 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.067169 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.067169 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:52.067169 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.067169 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.068711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.069331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:52.069673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:52.070071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.070549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:52.070549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.070549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.070549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:52.070549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.070549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:52.073605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:52.073883 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:52.074001 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:52.074156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:52.074755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:52.074956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.075555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:52.075555 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.075555 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:52.075555 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.075555 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.075555 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:52.075555 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.075555 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:52.076433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:52.076778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.076778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:52.076778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.076778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:52.076778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.076778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.076778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:52.076778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.076778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.077373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:52.077685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.077932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.077932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:52.077932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.077932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:52.078943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.079425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:52.079756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.079756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:52.079756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.079756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.079756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:52.079756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.079756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:52.087189 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:52.087823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:52.088016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:52.088266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:52.089392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:52.089681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.090282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:42:52.090282 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.090282 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:52.090282 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.090282 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.090282 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:52.090282 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.090282 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:52.091508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:52.091904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.091904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:52.091904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.091904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:52.091904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.091904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.091904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:52.091904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.091904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:52.093077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.093077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:52.093077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:52.093077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:52.093077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.093077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:52.093077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:52.093077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:52.093077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:52.093077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.093077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:52.093394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.093752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.093752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:52.093752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.093752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.095025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:52.095673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.096145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.096145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:52.096145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.096145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:52.150758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:52.151455 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:52.151592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:52.151811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:52.152407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:52.152706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:52.152824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.153393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:42:52.153393 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.153393 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:52.153393 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.153393 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.153393 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:52.153393 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.153393 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.154292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.154547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:52.154790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.154790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:52.155288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:52.155613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:52.155945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:52.155945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:52.155945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:57.010984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:57.011547 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:57.011547 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:57.012115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:57.055463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:57.055770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.056471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:42:57.056471 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.056471 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:57.056471 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.056471 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.056471 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:57.056471 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.056471 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.057380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.057826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:57.058347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:57.060975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.061367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:57.061367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.061367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.061367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:57.061367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.061367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:57.064422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:57.065950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:57.066176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:57.067309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:57.067589 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:57.067785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.068525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:42:57.068525 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.068525 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:57.068525 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.068525 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.068525 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:57.068525 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.068525 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.070475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.070946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:57.071367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.071367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:57.072417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.073182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.074258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:57.074258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.074258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:57.074258 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:57.075075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:57.075075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:57.075075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:42:57.075727 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:57.075727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.076449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:42:57.076449 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.076449 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:57.076449 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.076449 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.076449 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:57.076449 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.076449 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:57.077400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:57.077897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.077897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:57.077897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.077897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:57.077897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.077897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.077897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:57.077897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.077897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.078502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:57.079542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.079922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.079922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:57.079922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.079922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:57.080484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.081050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:57.081608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.081608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:57.081608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.081608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.081608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:57.081608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.081608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:57.087646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:57.088448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:57.088448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:57.088448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:57.088880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:57.089307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.092388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:42:57.092901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.093495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.093826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:57.094112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:57.094612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.095159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:57.095159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.095159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:57.095159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.095159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.095159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:57.095159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.095159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:42:57.151768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:42:57.152995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:42:57.152995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:42:57.153263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:42:57.154181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:57.154737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:42:57.155015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.156158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:42:57.156158 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.156158 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:42:57.156158 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.156158 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.156158 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:42:57.156158 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.156158 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.157840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.158626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:42:57.159270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:42:57.160064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:42:57.161178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:42:57.161178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:42:57.161178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:42:57.161178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:42:57.161178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:42:57.161178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:02.037505 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:02.037845 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:02.038578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:02.038842 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:02.065675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:02.067440 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:02.067549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:02.069886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:02.071406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:02.071406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.071406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:43:02.071406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.071406 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:02.071406 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.071406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.071406 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:02.071406 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.071406 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.077281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.077887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:02.078355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:02.078863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.079468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:02.079468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.079468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.079468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:02.079468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.079468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:02.079468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:02.079468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:02.080987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:02.081094 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:02.083678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:02.083998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.085385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:43:02.085385 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.085385 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:02.085385 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.085385 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.085385 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:02.085385 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.085385 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:02.092969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:02.093573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.093573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:02.093573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.093573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:02.093573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.093573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.093573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:02.093573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.093573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:02.097887 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.014 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.099456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:02.100096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.100667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.100667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:02.100667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.100667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:02.103013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.103462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:02.104372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.104372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:02.104372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.104372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.104372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:02.104372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.104372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:02.104372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:02.106083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:02.106083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:02.106728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:02.106987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.110560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:02.110560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.110560 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:02.110560 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.110560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.110560 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:02.110560 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.110560 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:02.111991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.112453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:02.112453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.112453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:02.112453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.112453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.112453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:02.112453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.112453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:02.117628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.118092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.118454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:02.118454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.118454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:02.118454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:02.118454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.120015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:02.120514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.120974 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_fdb_aging_settings failed after 0.083 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.122730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.123083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:02.123437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.123437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:02.125891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.126732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.127187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:02.127187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.127187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:02.152747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:02.153380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:02.153469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:02.153638 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:02.154342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:02.154661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:02.154833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.155747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:43:02.155747 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.155747 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:02.155747 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.155747 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.155747 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:02.155747 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.155747 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.156837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.157233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:02.157607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.157607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:02.157918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:02.158386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:02.158916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:02.158916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:02.158916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:07.037911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:07.038851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:07.038965 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:07.040351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:07.074037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:07.074411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:07.074701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:07.076300 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:07.077755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:07.077755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.078669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:07.079025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:07.079178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:07.079422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:07.079771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:07.079771 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.079771 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:07.079771 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.079771 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.079771 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:07.079771 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.079771 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.086350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.087119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:07.087770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.087770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:07.088930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.090054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.091013 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:07.092587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:07.093796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:07.094326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.094326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:07.094326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.094326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:07.094326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.094326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.094326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:07.094326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.094326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:07.094326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task set_fip_distributed_flag failed after 0.015 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.095500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:07.095912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.096350 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:07.097159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.097764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:07.098361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.098361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:07.098361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.098361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.098361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:07.098361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.098361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.098724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.099184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:07.099575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.099575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:07.099575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:07.101010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.101593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.102188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:07.102188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.102188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:07.102188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:07.102188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:07.102188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:07.102792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:07.102792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.103303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:43:07.103303 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.103303 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:07.103303 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.103303 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.103303 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:07.103303 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.103303 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:07.106945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:07.108698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.108698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:07.108698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.108698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:07.108698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.108698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.108698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:07.108698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.108698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:07.110281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.110281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:07.110281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:07.110281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:07.110281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.110281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:07.110281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:07.110281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:07.110281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:07.110281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.110281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:07.113102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:07.113694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:07.114041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.114457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:07.114457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:07.114457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.114457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:07.114457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.114457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:07.114457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.114457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.114457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:07.114457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.114457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:07.153531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:07.154041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:07.154148 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:07.154272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:07.154970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:07.155283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:07.155507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.158536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:07.158536 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.158536 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:07.158536 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.158536 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.158536 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:07.158536 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.158536 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.159398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:07.160111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:07.160640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:07.161228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:07.161861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:07.161861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:07.161861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:12.038695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:12.042146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:12.042347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:12.043682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:12.099392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.102385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.103707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:12.105334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:12.106228 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:12.106737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:12.106737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:12.106737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:12.106737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:12.106737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:12.106737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:12.107466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.108371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:12.108371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:12.108371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:12.108473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:43:12.108473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.108473 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:12.108473 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.108473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.108473 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:12.108473 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.108473 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:12.108986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.111749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.112072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:12.112371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.112371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:12.112671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:12.114361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:12.116150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.116523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.116875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:12.116875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.116875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:12.116875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:12.116875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.117926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:43:12.117926 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.117926 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:12.117926 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.117926 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.117926 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:12.117926 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.117926 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:12.118826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:12.119398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.119398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:12.119398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.119398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:12.119398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.119398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.119398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:12.119398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.119398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:12.119398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:12.119687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task set_fip_distributed_flag failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.121287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:12.121694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Traceback (most recent call last):
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.121980 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:12.123503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:12.123943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.123943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:12.123943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.123943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:12.123943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.123943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.123943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:12.123943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.123943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:12.125343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] OVN maintenance task update_ha_failover failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.125343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:12.125343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:12.125343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:12.125343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.125343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:12.125343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:12.125343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:12.125343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:12.125343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.125343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:12.125715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.126055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.126055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:12.126055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.126055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:12.127398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.127722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:12.128108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.128108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:12.128108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.128108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.128108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:12.128108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.128108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.128883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:12.129276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.129734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.129734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:12.129734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.129734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:12.154248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:12.154771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:12.154862 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:12.155033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:12.155696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:12.155926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:12.156059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.156662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:12.156662 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.156662 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:12.156662 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.156662 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.156662 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:12.156662 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.156662 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.157652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:12.158044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.158418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:12.158661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:12.159157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:12.159691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:12.159691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:17.042285 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:17.043555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:17.043709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:17.044707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:17.082260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:17.082471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.083123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:17.083123 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.083123 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:17.083123 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.083123 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.083123 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:17.083123 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.083123 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.084136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.084468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:17.084777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.084777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:17.085312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.086173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.086608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:17.086608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.086608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:17.102667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:17.103401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:17.103583 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:17.104859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:17.105345 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:17.105462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.106902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:17.108377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:17.108377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:17.108377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:17.108857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:17.109923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Traceback (most recent call last):
Apr 21 13:43:17.109923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.109923 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:17.109923 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.109923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.109923 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:17.109923 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.109923 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:17.110526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.111739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.114560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.114830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:17.115126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.115126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:17.115126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:17.115528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Traceback (most recent call last):
Apr 21 13:43:17.115528 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.115528 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:17.115528 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.115528 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.115528 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:17.115528 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.115528 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:17.117883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.118259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.118669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:17.118669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.118669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:17.118669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:17.119987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:17.120363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.120363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:17.120363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.120363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:17.120363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.120363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.120363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:17.120363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.120363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:17.122458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] OVN maintenance task set_fip_distributed_flag failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.124252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:17.124620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.125190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.125190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:17.125190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.125190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:17.125190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:17.125190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-32a5947d-082d-419b-afbe-4fb623c55823 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:17.126711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.127045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.127425 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:17.128037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:17.128369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.128369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:17.128369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.128369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:17.128369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.128369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.128369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:17.128369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.128369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:17.128833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_ha_failover failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.128833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:17.128833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:17.128833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:17.128833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.128833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:17.128833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:17.128833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:17.128833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:17.128833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.128833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:17.129068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.129354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.129354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:17.129354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.129354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.130078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:17.130450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.131055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.131055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:17.131055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.131055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:17.154986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:17.156313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:17.156313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:17.156313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:17.156313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:17.157017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:17.157124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.157914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:17.157914 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.157914 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:17.157914 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.157914 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.157914 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:17.157914 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.157914 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.158800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:17.159546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:17.160095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:17.161034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:17.161874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:17.161874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:17.161874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:22.043818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:22.044707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:22.044707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:22.045564 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:22.089248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:22.090244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.090244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:22.090244 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.090244 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:22.090244 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.090244 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.090244 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:22.090244 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.090244 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.091096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.091548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:22.091895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.091895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:22.092373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.092828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.093243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:22.093243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.093243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:22.104111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:22.104111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:22.104111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:22.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:22.106139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:22.106455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.107366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:22.107366 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.107366 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:22.107366 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.107366 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.107366 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:22.107366 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.107366 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.109721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.109997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:22.110248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.110248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:22.110248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.115247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.115769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:22.116117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.116117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:22.116117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:22.116117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:22.116117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:22.116117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:22.116117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:22.116307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:22.116307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:22.116307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:22.116307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.117204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:22.117487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.118096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:43:22.118096 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.118096 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:22.118096 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.118096 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.118096 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:22.118096 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.118096 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:22.119157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:22.119507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.119507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:22.119507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.119507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:22.119507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.119507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.119507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:22.119507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.119507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:22.119703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:43:22.119703 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.119703 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:22.119703 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.119703 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.119703 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:22.119703 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.119703 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.120854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:22.121116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.121374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.121374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:22.121374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.121374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:22.121856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:22.122134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.122134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:22.122134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.122134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:22.122134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.122134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.122134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:22.122134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.122134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:22.123044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.123373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:22.123696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.123696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:22.123696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.123696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.123696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:22.123696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.123696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:22.124040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.124040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:22.124040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:22.124040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:22.124040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.124040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:22.124040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:22.124040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:22.124040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:22.124040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.124040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:22.124268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.124512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.124512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:22.124512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.124512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.125177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:22.125514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.125849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.125849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:22.125849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.125849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:22.155846 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:22.156629 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:22.156740 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:22.156934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:22.157618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:22.157970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:22.158206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.159079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:22.159079 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.159079 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:22.159079 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.159079 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.159079 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:22.159079 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.159079 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.160977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:22.161354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.161705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:22.163259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:22.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:22.164166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:22.164166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:27.050428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:27.052065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:27.052065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:27.053870 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:27.101489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:27.101783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.102733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:27.102733 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.102733 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:27.102733 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.102733 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.102733 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:27.102733 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.102733 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.104128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.104863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:27.105766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:27.106599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.107634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:27.107634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.107634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.107634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:27.107634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.107634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:27.107634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:27.107634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:27.107634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:27.107634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:27.108287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:27.108287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.108670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:27.108670 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.108670 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:27.108670 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.108670 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.108670 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:27.108670 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.108670 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.109786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.110436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:27.110816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.110816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:27.111061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.111541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.112016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:27.112016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.112016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:27.112016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:27.112016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:27.112016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:27.112222 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:27.115473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.115996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:27.116529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:27.116883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.117263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.118159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:27.118699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:27.119000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:27.119000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:27.119000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:27.119000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.120547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:27.120547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.120547 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:27.120547 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.120547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.120547 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:27.120547 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.120547 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:27.122345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.122875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:27.122875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.122875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:27.122875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.122875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.122875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:27.122875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.122875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:27.123712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.124151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.124555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:27.124555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.124555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.125609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:27.126107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.126601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.126601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:27.126601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.126601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:27.156688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:27.157255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:27.157404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:27.157612 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:27.158351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:27.158900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:27.159256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.160182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:27.160182 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.160182 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:27.160182 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.160182 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.160182 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:27.160182 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.160182 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.161536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:27.162184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.162786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:27.163002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:27.163895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:27.164938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:27.164938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:32.051297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:32.052129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:32.052311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:32.053177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:32.096125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:32.096424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.097285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:32.097285 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.097285 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:32.097285 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.097285 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.097285 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:32.097285 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.097285 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.098351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.099163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:32.099962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:32.100915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.102139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:32.102139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.102139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.102139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:32.102139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.102139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:32.107025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:32.107596 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:32.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:32.108722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:32.108959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:32.109174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.109879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:43:32.109879 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.109879 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:32.109879 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.109879 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.109879 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:32.109879 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.109879 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.111711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.112800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:32.113843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.113843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:32.113843 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:32.114568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.115117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.116319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:32.116319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.116319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:32.116319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:32.116319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:32.116319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:32.116319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:32.116319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.117031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:43:32.117031 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.117031 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:32.117031 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.117031 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.117031 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:32.117031 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.117031 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:32.118997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:32.119766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.119766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:32.119766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.119766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:32.119766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.119766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.119766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:32.119766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.119766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:32.119766 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.121616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:32.121998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.122383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.122383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:32.122383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.122383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:32.122383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:32.122383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:32.123292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.123752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:32.124310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.124310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:32.124310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.124310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.124310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:32.124310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.124310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:32.124310 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:32.125935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:32.126371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.127320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Traceback (most recent call last):
Apr 21 13:43:32.127320 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.127320 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:32.127320 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.127320 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.127320 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:32.127320 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.127320 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:32.128583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:32.128986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.128986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:32.128986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.128986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:32.128986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.128986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.128986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:32.128986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.128986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:32.129627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.129627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:32.129627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:32.129627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:32.129627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.129627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:32.129627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:32.129627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:32.129627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:32.129627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.129627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:32.130069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.130481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.130481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:32.130481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.130481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.131145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:32.131641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.132104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.132104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:32.132104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.132104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:32.178117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:32.178117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:32.178117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:32.178117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:32.178117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:32.178117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:32.181046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:32.181416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:32.181859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:32.182455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:32.182455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:32.182455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:32.182455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:32.182455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:32.182455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:32.182455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:32.182455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:32.182455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:37.051930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:37.052585 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:37.052646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:37.053773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:37.089801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:37.090062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.090773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:37.090773 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.090773 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:37.090773 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.090773 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.090773 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:37.090773 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.090773 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.091831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.092123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:37.092387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.092387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:37.093853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.094301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.094786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:37.094786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.094786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:37.107698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:37.108390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:37.108520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:37.109419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:37.109720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:37.109891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.111893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:37.111893 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.111893 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:37.111893 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.111893 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.111893 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:37.111893 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.111893 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.113164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.113766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:37.114340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.114340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.114458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.115250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:37.116025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.116025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:37.116025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:37.116025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:37.116025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:37.116025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:37.116856 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:37.116856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.117036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:37.117036 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.117036 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:37.117036 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.117036 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.117036 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:37.117036 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.117036 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:37.117780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.118817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:37.119753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.120599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:37.121590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.122502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:37.123026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:37.123026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:37.123026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:37.123026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:37.123026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:37.123026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.123647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Traceback (most recent call last):
Apr 21 13:43:37.123647 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.123647 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:37.123647 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.123647 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.123647 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:37.123647 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.123647 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:37.124175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:37.125100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:37.126008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:37.126702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:37.127334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.127816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:37.164297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:37.164671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:37.164755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:37.164905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:37.165489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:37.165726 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:37.165880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.166456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:37.166456 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.166456 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:37.166456 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.166456 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.166456 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:37.166456 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.166456 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.167238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:37.167885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:37.168645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:37.169436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:37.171603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:37.171603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:37.171603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:42.052536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:42.053449 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:42.053449 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:42.054151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:42.058200 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:43:42.087225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:42.087643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.088677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:42.088677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.088677 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:42.088677 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.088677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.088677 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:42.088677 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.088677 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.089568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.089999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:42.090405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:42.090875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.091522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:42.091522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.091522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.091522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:42.091522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.091522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:42.108628 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:42.109234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:42.109234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:42.109740 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:42.109986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:42.110154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.110734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:42.110734 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.110734 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:42.110734 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.110734 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.110734 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:42.110734 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.110734 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.111420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.111706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:42.111972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.111972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.112121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.112487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:42.113450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.113450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:42.115352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:42.116778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:42.116930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:42.117046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:42.117611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:42.117831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.118412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:42.118412 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.118412 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:42.118412 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.118412 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.118412 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:42.118412 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.118412 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:42.119562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.120458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:42.120458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.120458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:42.120458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.120458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.120458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:42.120458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.120458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.120627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:42.120904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.121180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.121180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:42.121180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.121180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.123018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:42.123396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.123751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.123751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:42.123751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.123751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:42.123751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:42.123751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:42.123751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:42.123751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:42.124455 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:42.124675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.126081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:42.126081 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.126081 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:42.126081 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.126081 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.126081 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:42.126081 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.126081 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:42.127041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.127718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:42.127718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.127718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:42.127718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.127718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.127718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:42.127718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.127718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:42.128471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.129012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.129273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:42.129273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.129273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.130017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:42.130403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.130740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.130740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:42.130740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.130740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:42.164983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:42.165278 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:42.165339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:42.165470 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:42.165863 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:42.166128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:42.166298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.166872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:42.166872 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.166872 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:42.166872 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.166872 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.166872 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:42.166872 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.166872 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.167536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:42.167819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.168050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:42.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:42.169181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:42.169543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:42.169543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:47.054234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:47.055340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:47.055340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:47.056994 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:47.088063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:47.088430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.089166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:47.089166 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.089166 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:47.089166 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.089166 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.089166 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:47.089166 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.089166 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.090428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.090958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:47.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.091547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:47.092812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.096536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.097189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:47.097189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.097189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:47.109345 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:47.109834 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:47.109986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:47.111125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:47.111677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:47.111910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.112803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:47.112803 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.112803 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:47.112803 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.112803 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.112803 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:47.112803 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.112803 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.114119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.115137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.115676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:47.116450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.117830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.117830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:47.117830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.117830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:47.117830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:47.117830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:47.117830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:47.117830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:47.118560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:47.118875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.119535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:43:47.119535 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.119535 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:47.119535 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.119535 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.119535 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:47.119535 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.119535 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:47.120730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:47.121831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.123045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.123402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:47.126743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.127097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:47.128109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.128109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:47.128109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.128109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.128109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:47.128109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.128109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:47.128109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:47.128109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:47.128109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:47.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:47.129329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:47.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:47.131470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:47.131798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.131798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:47.131798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.131798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:47.131798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.131798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.131798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:47.131798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.131798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.135786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:47.136325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.136830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.136830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:47.136830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.136830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:47.165690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:47.166135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:47.166250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:47.166539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:47.166928 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:47.167178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:47.167282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.167857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:47.167857 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.167857 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:47.167857 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.167857 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.167857 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:47.167857 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.167857 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.168588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:47.168946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:47.169328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:47.169814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:47.170537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:47.170537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:47.170537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:52.054972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:52.056832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:52.056832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:52.056832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:52.095969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:52.096399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.097079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:52.097079 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.097079 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:52.097079 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.097079 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.097079 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:52.097079 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.097079 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.098064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.098708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:52.099475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:52.100269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.100919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:52.100919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.100919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.100919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:52.100919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.100919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:52.109996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:52.110815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:52.110913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:52.111884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:52.112165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:52.112372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.113023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:52.113023 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.113023 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:52.113023 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.113023 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.113023 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:52.113023 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.113023 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.114033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.114734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.115395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:52.116168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.117066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.117066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:52.117066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.117066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:52.117066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:52.117567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:52.117665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:52.117824 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:52.118472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:52.118685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.119232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:43:52.119232 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.119232 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:52.119232 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.119232 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.119232 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:52.119232 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.119232 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:52.120179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:52.120718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.120718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:52.120718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.120718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:52.120718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.120718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.120718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:52.120718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.120718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.121227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:52.121738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:52.122926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.123535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:52.124378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:52.124882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:52.124882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:52.125003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:52.125678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:52.125678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.126423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Traceback (most recent call last):
Apr 21 13:43:52.126423 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.126423 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:52.126423 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.126423 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.126423 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:52.126423 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.126423 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:52.127137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:52.127723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.127723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:52.127723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.127723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:52.127723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.127723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.127723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:52.127723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.127723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:52.128386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.128386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:52.128386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:52.128386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:52.128386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.128386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:52.128386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:52.128386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:52.128386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:52.128386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.128386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:52.128885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.129486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.129486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:52.129486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.129486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.130415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:52.131525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.133387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.133387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:52.133387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.133387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:52.166292 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:52.166832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:52.167002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:52.167319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:52.168001 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:52.168263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:52.168405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.169119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:52.169119 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.169119 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:52.169119 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.169119 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.169119 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:52.169119 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.169119 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.171232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:52.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.172305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:52.172409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:52.172968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:52.173626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:52.173626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:57.055508 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:57.056556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:57.056556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:57.057347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:57.092954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:57.093286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.094094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:43:57.094094 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.094094 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:57.094094 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.094094 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.094094 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:57.094094 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.094094 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.095067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.098593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:57.099121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:57.099633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.100185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:57.100185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.100185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.100185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:57.100185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.100185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:57.111828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:57.112375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:57.112518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:57.113437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:57.113715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:57.113867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.114454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:57.114454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.114454 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:57.114454 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.114454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.114454 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:57.114454 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.114454 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.116024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.116718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:57.117357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.117357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.118484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.119430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:57.120301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.120301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:57.120301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:57.120301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:57.120301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:57.120301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:43:57.120301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:57.120837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.121184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:43:57.121184 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.121184 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:57.121184 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.121184 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.121184 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:57.121184 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.121184 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:57.122109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:57.122904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.122904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:57.122904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.122904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:57.122904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.122904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.122904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:57.122904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.122904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.123508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:57.124135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:57.124811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.125703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:57.126644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:57.127072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:57.127072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:57.127072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:57.127072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.127411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:43:57.127411 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.127411 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:57.127411 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.127411 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.127411 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:57.127411 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.127411 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:43:57.128316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:43:57.128809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.128809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:43:57.128809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.128809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:43:57.128809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.128809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.128809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:43:57.128809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.128809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:43:57.130077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.130077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:57.130077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:57.130077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:57.130077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.130077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:57.130077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:57.130077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:57.130077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:57.130077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.130077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:57.130469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.130870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.130870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:57.130870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.130870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.131962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:57.132494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.133239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.133239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:57.133239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.133239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:43:57.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:43:57.167439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:43:57.167599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:43:57.167689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:43:57.168128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:57.168515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:43:57.168747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.169503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:43:57.169503 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.169503 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:43:57.169503 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.169503 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.169503 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:43:57.169503 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.169503 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.170490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:43:57.171266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:43:57.171845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:43:57.173073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:43:57.174137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:43:57.174137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:43:57.174137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:02.055950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:02.056498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:02.056904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:02.057548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:02.095945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:02.096318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.096893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:02.096893 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.096893 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:02.096893 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.096893 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.096893 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:02.096893 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.096893 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.097693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.098120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:02.098507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:02.098976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.099585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:02.099585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.099585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.099585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:02.099585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.099585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:02.112546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:02.112994 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:02.113085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:02.115571 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:02.115785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:02.115918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.116477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:02.116477 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.116477 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:02.116477 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.116477 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.116477 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:02.116477 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.116477 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.117803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.118154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:02.118413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.118413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.119520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.119864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:02.120222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.120222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:02.120222 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:02.120344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:02.120475 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:02.120604 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:02.121009 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:02.121261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.125574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:02.125574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.125574 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:02.125574 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.125574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.125574 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:02.125574 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.125574 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:02.126846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:02.127266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.127266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:02.127266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.127266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:02.127266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.127266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.127266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:02.127266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.127266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.127921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:02.128202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.128543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.128543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:02.128543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.128543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:02.129341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.130029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:02.130550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.130550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:02.130550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.130550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.130550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:02.130550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.130550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:02.130550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:02.130550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:02.130550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:02.130891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:02.130891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:02.130891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.131311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:44:02.131311 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.131311 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:02.131311 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.131311 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.131311 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:02.131311 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.131311 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:02.132330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:02.132619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.132619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:02.132619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.132619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:02.132619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.132619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.132619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:02.132619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.132619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:02.133389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.133389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:02.133389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:02.133389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:02.133389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.133389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:02.133389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:02.133389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:02.133389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:02.133389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.133389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:02.133657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.133919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.133919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:02.133919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.133919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.134579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:02.134960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.135365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.135365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:02.135365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.135365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:02.167837 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:02.168049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:02.168089 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:02.168252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:02.168672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:02.168967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:02.169137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.169830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:02.169830 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.169830 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:02.169830 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.169830 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.169830 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:02.169830 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.169830 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.170822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:02.171124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.171428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:02.171814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:02.172180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:02.172546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:02.172546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:07.056586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:07.057523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:07.057523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:07.057915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:07.095949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:07.096283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.096990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:07.096990 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.096990 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:07.096990 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.096990 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.096990 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:07.096990 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.096990 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.099158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.099441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:07.099694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:07.099984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.100346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:07.100346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.100346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.100346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:07.100346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.100346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:07.113346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:07.113947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:07.114027 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:07.114944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:07.115237 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:07.115417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.116028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:07.116028 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.116028 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:07.116028 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.116028 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.116028 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:07.116028 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.116028 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.117407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.117693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:07.117939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.117939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.118722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.119068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:07.119463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.119463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:07.120123 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:07.120786 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:07.120978 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:07.121127 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:07.121554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:07.121775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.124514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:07.124514 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.124514 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:07.124514 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.124514 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.124514 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:07.124514 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.124514 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:07.125528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:07.125882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.125882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:07.125882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.125882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:07.125882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.125882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.125882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:07.125882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.125882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.126616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:07.126906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.127188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.127188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:07.127188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.127188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:07.128022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.128366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:07.128729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.128729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:07.128729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.128729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.128729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:07.128729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.128729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:07.129909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:07.132064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:07.132233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:07.132357 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:07.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:07.133181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.134757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:44:07.134757 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.134757 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:07.134757 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.134757 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.134757 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:07.134757 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.134757 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:07.135839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:07.136128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.136128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:07.136128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.136128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:07.136128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.136128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.136128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:07.136128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.136128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:07.137964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.137964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:07.137964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:07.137964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:07.137964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.137964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:07.137964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:07.137964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:07.137964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:07.137964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.137964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:07.138228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.138508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.138508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:07.138508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.138508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.139673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:07.140018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.140392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.140392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:07.140392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.140392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:07.169079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:07.169079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:07.169079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:07.169079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:07.169617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:07.170058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:07.170325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.172562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:07.172562 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.172562 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:07.172562 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.172562 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.172562 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:07.172562 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.172562 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.173459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:07.173881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:07.174246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:07.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:07.175224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:07.175224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:07.175224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:12.057172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:12.060184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:12.060184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:12.060184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:12.101086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:12.101412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.102197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:12.102197 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.102197 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:12.102197 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.102197 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.102197 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:12.102197 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.102197 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.103406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.103762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:12.104076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.104076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:12.104438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.104852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.105302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:12.105302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.105302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:12.113941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:12.115008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:12.115008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:12.115476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:12.115694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:12.115811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:12.117363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.117927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:12.118374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:12.118823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:12.119347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.119347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:12.119347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.119347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:12.119347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.119347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.119347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:12.119347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.119347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:12.120639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:12.121103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:12.121190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:12.121291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:12.122642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:12.122832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.123395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:12.123395 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.123395 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:12.123395 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.123395 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.123395 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:12.123395 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.123395 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:12.124076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:12.124565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.124565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:12.124565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.124565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:12.124565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.124565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.124565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:12.124565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.124565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.124834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:12.125188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:12.125587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.125949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.126498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:12.132087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:12.132677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:12.132880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:12.132967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:12.133618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:12.133863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.134485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Traceback (most recent call last):
Apr 21 13:44:12.134485 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.134485 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:12.134485 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.134485 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.134485 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:12.134485 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.134485 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:12.135680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:12.136095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.136095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:12.136095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.136095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:12.136095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.136095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.136095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:12.136095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.136095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:12.137026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.137026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:12.137026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:12.137026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:12.137026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.137026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:12.137026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:12.137026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:12.137026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:12.137026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.137026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:12.137412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.137837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.137837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:12.137837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.137837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.138178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:12.138787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.139295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.139295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:12.139295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.139295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:12.168945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:12.170019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:12.170122 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:12.170346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:12.170936 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:12.171278 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:12.171471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.172226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:12.172226 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.172226 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:12.172226 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.172226 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.172226 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:12.172226 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.172226 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.173690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:12.174096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.174466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:12.174841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:12.175389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:12.175818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:12.175818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:17.058736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:17.059711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:17.059711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:17.062728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:17.104313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:17.104313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.104528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:17.104528 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.104528 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:17.104528 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.104528 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.104528 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:17.104528 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.104528 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.106044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.106369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:17.106771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.106771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:17.107093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.107507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.107909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:17.107909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.107909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:17.116497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:17.116497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:17.116497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:17.116497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:17.116497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:17.116497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.116943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:17.116943 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.116943 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:17.116943 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.116943 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.116943 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:17.116943 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.116943 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.117735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.118111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.118466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:17.118901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.119583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.119583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:17.119583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.119583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:17.120933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:17.121592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:17.121774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:17.121936 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:17.122329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:17.122939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.123590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:17.123590 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.123590 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:17.123590 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.123590 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.123590 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:17.123590 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.123590 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:17.124549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:17.124885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.124885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:17.124885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.124885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:17.124885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.124885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.124885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:17.124885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.124885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.125395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:17.125653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.125914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.125914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:17.125914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.125914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:17.126553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.126883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:17.127268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.127268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:17.127268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.127268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.127268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:17.127268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.127268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:17.135187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:17.135557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:17.135688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:17.135823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:17.137017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:17.137017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:44:17.137243 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.137243 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:17.137243 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.137243 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.137243 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:17.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.137243 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:17.138174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:17.138493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.138493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:17.138493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.138493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:17.138493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.138493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.138493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:17.138493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.138493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:17.139170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.139170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:17.139170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:17.139170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:17.139170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.139170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:17.139170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:17.139170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:17.139170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:17.139170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.139170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:17.139836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.140137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.140137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:17.140137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.140137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.142409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:17.142795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.143144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.143144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:17.143144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.143144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:17.171754 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:17.172238 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:17.172318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:17.172508 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:17.172998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:17.174091 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:17.174091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.174276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:17.174276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.174276 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:17.174276 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.174276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.174276 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:17.174276 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.174276 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.175574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:17.175995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.176487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:17.176570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:17.177089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:17.177530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:17.177530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:22.059493 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:22.060081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:22.060081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:22.061150 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:22.098534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:22.098534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.098860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:22.098860 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.098860 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:22.098860 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.098860 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.098860 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:22.098860 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.098860 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.100051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.100784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:22.101542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:22.102432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.103397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:22.103397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.103397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.103397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:22.103397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.103397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:22.115180 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:22.115906 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:22.116092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:22.118496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:22.118496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:22.118496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.118751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:22.118751 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.118751 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:22.118751 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.118751 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.118751 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:22.118751 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.118751 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.119763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.120468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.121164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:22.122298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.123167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.123167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:22.123167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.123167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:22.123696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:22.123926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:22.124114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:22.124350 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:22.125544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:22.125544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.125887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:22.125887 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.125887 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:22.125887 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.125887 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.125887 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:22.125887 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.125887 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:22.127032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:22.127741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.127741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:22.127741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.127741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:22.127741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.127741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.127741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:22.127741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.127741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.128272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:22.128791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.129364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.129364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:22.129364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.129364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:22.129965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.130640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:22.131164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.131164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:22.131164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.131164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.131164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:22.131164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.131164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:22.135646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:22.136053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:22.136143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:22.136284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:22.137275 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:22.137275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.138463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:22.138463 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.138463 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:22.138463 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.138463 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.138463 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:22.138463 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.138463 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:22.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.139812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:22.139812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.139812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:22.139812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.139812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.139812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:22.139812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.139812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:22.140172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.140552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.140909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:22.140909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.140909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.143555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:22.144196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.144764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.144764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:22.144764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.144764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:22.172582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:22.172815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:22.172909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:22.173084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:22.173084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:22.173084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:22.173084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.175121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:22.175121 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.175121 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:22.175121 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.175121 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.175121 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:22.175121 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.175121 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.176449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:22.176849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.177269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:22.177472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:22.178055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:22.178700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:22.178700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:27.062367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:27.063153 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:27.063153 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:27.064412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:27.104552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:27.104778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.105463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:27.105463 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.105463 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:27.105463 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.105463 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.105463 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:27.105463 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.105463 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.106399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.107036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:27.107725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:27.108478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.109369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:27.109369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.109369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.109369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:27.109369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.109369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:27.115838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:27.116553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:27.116649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:27.117577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:27.117805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:27.117925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.118584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:27.118584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.118584 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:27.118584 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.118584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.118584 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:27.118584 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.118584 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.120352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.121288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.121817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:27.122430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.123186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.123186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:27.123186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.123186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:27.123958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:27.124758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:27.124946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:27.125138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:27.125645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:27.125878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.126427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:27.126427 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.126427 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:27.126427 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.126427 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.126427 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:27.126427 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.126427 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:27.127353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:27.127728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.127728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:27.127728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.127728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:27.127728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.127728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.127728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:27.127728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.127728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.128552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:27.128892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.129273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.129273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:27.129273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.129273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:27.130188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.130494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:27.131277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.131277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:27.131277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.131277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.131277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:27.131277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.131277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:27.136106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:27.136523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:27.136709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:27.136925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:27.137632 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:27.138040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.139011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:44:27.139011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.139011 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:27.139011 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.139011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.139011 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:27.139011 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.139011 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:27.140151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:27.140806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.140806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:27.140806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.140806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:27.140806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.140806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.140806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:27.140806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.140806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:27.141186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.141186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:27.141186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:27.141186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:27.141186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.141186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:27.141186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:27.141186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:27.141186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:27.141186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.141186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:27.141652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.142194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.142194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:27.142194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.142194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.142407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:27.143516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.144365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.144365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:27.144365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.144365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:27.173079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:27.173422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:27.173504 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:27.173697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:27.174375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:27.174942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:27.175370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.176370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:27.176370 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.176370 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:27.176370 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.176370 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.176370 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:27.176370 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.176370 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.177284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:27.177978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:27.178634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:27.179476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:27.180270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:27.180270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:27.180270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:32.062988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:32.063933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:32.064056 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:32.065673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:32.102784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:32.103159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.104112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:32.104112 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.104112 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:32.104112 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.104112 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.104112 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:32.104112 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.104112 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.105104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.106377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:32.106972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:32.107727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.108700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:32.108700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.108700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.108700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:32.108700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.108700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:32.116591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:32.117174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:32.117304 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:32.118308 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:32.118707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:32.119171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.120246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:32.120246 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.120246 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:32.120246 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.120246 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.120246 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:32.120246 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.120246 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.121449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.121949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.122827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:32.123248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.123744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.123744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:32.123744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.123744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:32.124681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:32.125586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:32.125764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:32.125929 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:32.126778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:32.127020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.127637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:32.127637 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.127637 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:32.127637 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.127637 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.127637 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:32.127637 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.127637 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:32.128703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:32.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:32.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:32.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:32.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.131948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:32.132342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.132589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.132589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:32.132589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.132589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:32.133352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.133663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:32.133973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.133973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:32.133973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.133973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.133973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:32.133973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.133973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:32.136562 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:32.136899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:32.137051 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:32.137240 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:32.137809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:32.138067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.138708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:44:32.138708 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.138708 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:32.138708 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.138708 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.138708 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:32.138708 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.138708 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:32.139732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:32.140947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.140947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:32.140947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.140947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:32.140947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.140947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.140947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:32.140947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.140947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:32.141235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.141235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:32.141235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:32.141235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:32.141235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.141235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:32.141235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:32.141235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:32.141235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:32.141235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.141235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:32.141633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.141975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.141975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:32.141975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.141975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.142911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:32.143415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.144108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.144108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:32.144108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.144108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:32.173771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:32.174330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:32.174452 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:32.174669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:32.175319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:32.175765 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:32.176015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.176929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:32.176929 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.176929 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:32.176929 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.176929 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.176929 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:32.176929 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.176929 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.178038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:32.178652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.179205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:32.179435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:32.179921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:32.180416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:32.180416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:37.063780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:37.064938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:37.064938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:37.065094 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:37.102866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:37.103092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.103789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:37.103789 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.103789 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:37.103789 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.103789 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.103789 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:37.103789 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.103789 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.104702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.105286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:37.105824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:37.106361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.107335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:37.107335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.107335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.107335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:37.107335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.107335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:37.117115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:37.117561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:37.117648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:37.118398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:37.118630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:37.118695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.119265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:37.119265 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.119265 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:37.119265 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.119265 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.119265 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:37.119265 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.119265 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.120354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.120900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.121459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:37.122095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.123469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.123469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:37.123469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.123469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:37.125139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:37.125894 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:37.125988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:37.126102 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:37.126611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:37.126882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.127567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:37.127567 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.127567 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:37.127567 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.127567 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.127567 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:37.127567 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.127567 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:37.128249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:37.128858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.129501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.130117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:37.130803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.131486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:37.131988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.131988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:37.131988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.131988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.131988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:37.131988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.131988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:37.137828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:37.138188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:37.138353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:37.138578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:37.139039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:37.139368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.139918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:44:37.139918 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.139918 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:37.139918 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.139918 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.139918 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:37.139918 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.139918 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:37.140727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:37.141145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.141145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:37.141145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.141145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:37.141145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.141145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.141145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:37.141145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.141145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:37.141781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.141781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:37.141781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:37.141781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:37.141781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.141781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:37.141781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:37.141781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:37.141781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:37.141781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.141781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:37.142186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.142661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.142661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:37.142661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.142661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.143054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:37.143761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.144612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.144612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:37.144612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.144612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:37.174577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:37.175101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:37.175253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:37.175434 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:37.175914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:37.176224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:37.176399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.177121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:37.177121 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.177121 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:37.177121 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.177121 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.177121 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:37.177121 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.177121 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.178780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:37.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.179370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:37.181120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:37.181957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:37.183163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:37.183163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:42.065532 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:42.066941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:42.067089 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:42.068379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:42.070314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:44:42.106975 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:42.107271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.108124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:42.108124 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.108124 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:42.108124 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.108124 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.108124 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:42.108124 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.108124 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.110095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.110732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:42.111123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:42.112560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.113138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:42.113138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.113138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.113138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:42.113138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.113138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:42.117788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:42.118422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:42.118499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:42.119531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:42.119855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:42.120033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.120838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:42.120838 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.120838 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:42.120838 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.120838 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.120838 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:42.120838 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.120838 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.121765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.122154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.122675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:42.123166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.124197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.124197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:42.124197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.124197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:42.125851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:42.126562 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:42.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:42.126857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:42.127690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:42.127690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.128297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:42.128297 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.128297 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:42.128297 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.128297 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.128297 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:42.128297 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.128297 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:42.129369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:42.129728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.129728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:42.129728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.129728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:42.129728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.129728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.129728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:42.129728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.129728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.132196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:42.132503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.132763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.132763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:42.132763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.132763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:42.133392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.133745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:42.134089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.134089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:42.134089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.134089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.134089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:42.134089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.134089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:42.138492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:42.139570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:42.139784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:42.140005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:42.140792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:42.141158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.141663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:44:42.141663 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.141663 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:42.141663 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.141663 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.141663 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:42.141663 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.141663 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:42.143258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:42.143696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:42.144083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:42.144462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:42.144843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.145464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:42.175186 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:42.175980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:42.175980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:42.176334 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:42.176890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:42.177168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:42.179093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.179093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:42.179093 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.179093 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:42.179093 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.179093 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.179093 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:42.179093 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.179093 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.179908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:42.180407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.180784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:42.181441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:42.181990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:42.182568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:42.182568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:46.459414 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:46.460287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:46.460563 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:46.473794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 21 13:44:46.473978 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106267) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 21 13:44:46.474078 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Fixing resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:44:46.517684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, resource={'id': 'b57c058f-054e-4a2a-980f-6bfda2111e90', 'name': 'private', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['6a3993e6-b5ba-4812-baa3-b25db5880cd9', 'c97deb27-a1d6-4e03-b489-95abde8fd791'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:29:32Z', 'updated_at': '2026-04-21T13:29:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 24380}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.518342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3f6c78fa-b671-412d-bb34-5426a40d8bff, external_ids_update=None, columns={'tag': 24380}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.518342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.519045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Traceback (most recent call last):
Apr 21 13:44:46.519045 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.519045 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:46.519045 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.519045 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.519045 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:46.519045 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.519045 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:46.523143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Failed to fix resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.523143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:46.523143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:44:46.523143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:44:46.523143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:44:46.523143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:44:46.523143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:44:46.523143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:46.523143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.523143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.523143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.523662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.524309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:46.524309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.524309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:46.524309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Fixing resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:44:46.553430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d0607710-681d-444d-be26-dbeca5a07b46, resource={'id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'name': 'public', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-21T13:29:38Z', 'updated_at': '2026-04-21T13:29:48Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.553681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d0607710-681d-444d-be26-dbeca5a07b46, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.554008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-19f04b40-ed99-482a-8774-a76892a716d1, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.554188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.554843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Traceback (most recent call last):
Apr 21 13:44:46.554843 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.554843 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:46.554843 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.554843 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.554843 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:46.554843 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.554843 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:46.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Failed to fix resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:46.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:44:46.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:44:46.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:44:46.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:44:46.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:44:46.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:46.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.557156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.557541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:46.557541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.557541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:46.557541 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Fixing resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:44:46.589860 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, resource={'id': '345e31fe-ede8-4dd0-823e-e81dcbc5bef0', 'name': 'admin_net', 'tenant_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'project_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['bb271c4b-4172-418f-995c-f72abd439dbe'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:34:46Z', 'updated_at': '2026-04-21T13:34:47Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 22486}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.590143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.590250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-c1b4e94e-49ea-4c1f-8edf-2f1433fdff16, external_ids_update=None, columns={'tag': 22486}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.590366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.591001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Traceback (most recent call last):
Apr 21 13:44:46.591001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.591001 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:46.591001 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.591001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.591001 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:46.591001 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.591001 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:46.591909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Failed to fix resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.591909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:46.591909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:44:46.591909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:44:46.591909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:44:46.591909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:44:46.591909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:44:46.591909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:46.591909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.591909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.591909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.592449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.593002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:46.593002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.593002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:46.593002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Fixing resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:44:46.624871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, resource={'id': 'b364ad09-f70d-45d8-b9dd-8420f5f1c377', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['620e9559-2c15-4370-904f-f4677bc69542'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:35:24Z', 'updated_at': '2026-04-21T13:35:25Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 61006}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.625135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.625381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a100d4e8-7f11-40c9-8e4a-346c2c764f69, external_ids_update=None, columns={'tag': 61006}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.625456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.626164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Traceback (most recent call last):
Apr 21 13:44:46.626164 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.626164 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:46.626164 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.626164 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.626164 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:46.626164 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.626164 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:46.627475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Failed to fix resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.627475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:46.627475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:44:46.627475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:44:46.627475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:44:46.627475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:44:46.627475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:44:46.627475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:46.627475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.627475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.627475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.627828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.628225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:46.628225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.628225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:46.628225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Fixing resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:44:46.707333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, resource={'id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'enable_snat': True}, 'gw_port_id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-61m1f', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'ip_address': '172.24.5.49', 'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'}, {'ip_address': '2001:db8::162', 'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 3, 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04'}, resource_type=routers, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.707556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.707811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.709430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.709430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.709430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Traceback (most recent call last):
Apr 21 13:44:46.709430 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.709430 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:46.709430 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.709430 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.709430 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:46.709430 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.709430 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:46.709430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Unable to update router 06899d4c-a436-45e4-bed5-95dabe888f51. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.710498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Failed to fix resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.710498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:46.710498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:44:46.710498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:44:46.710498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:44:46.710498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:44:46.710498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 21 13:44:46.710498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 21 13:44:46.710498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 21 13:44:46.710498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 21 13:44:46.710498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:46.710892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:46.711344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:46.711344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:46.711344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.711344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:46.711344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.711344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.711344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:46.711344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.711344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:46.711344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Fixing resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:44:46.816913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, resource={'id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'name': '', 'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'project_id': '', 'mac_address': 'fa:16:3e:fb:71:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000168688', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.818922 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3 ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'neutron:network_name': 'neutron-d0607710-681d-444d-be26-dbeca5a07b46', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-06899d4c-a436-45e4-bed5-95dabe888f51'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.49/24', '2001:db8::162/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.818922 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, lrouter_port=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:46.818922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.819063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Traceback (most recent call last):
Apr 21 13:44:46.819063 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.819063 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:46.819063 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.819063 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.819063 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:46.819063 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.819063 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:46.819948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] Maintenance task: Failed to fix resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.819948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:46.819948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:44:46.819948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:44:46.819948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:44:46.819948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:44:46.819948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 21 13:44:46.819948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:46.819948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.819948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.819948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:46.820241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:46.820535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:46.820535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:46.820535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:46.820535 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7a84481e-78b0-402a-934b-65d7127700e5 None None] OVN maintenance task check_for_inconsistencies finished in 0.359 seconds
Apr 21 13:44:47.066921 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:47.067599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:47.067702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:47.068608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:47.109121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:47.109372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.110102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:47.110102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.110102 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:47.110102 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.110102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.110102 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:47.110102 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.110102 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.110918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.111434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:47.111788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:47.112274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.112827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:47.112827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.112827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.112827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:47.112827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.112827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:47.118423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:47.119511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:47.119511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:47.120379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:47.120709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:47.120919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.122876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:47.122876 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.122876 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:47.122876 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.122876 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.122876 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:47.122876 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.122876 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.124001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.124672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.124995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:47.126113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.126633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.126633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:47.126633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.126633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:47.126633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:47.127393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:47.127393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:47.127393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:47.127858 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:47.128091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.129165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:47.129165 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.129165 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:47.129165 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.129165 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.129165 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:47.129165 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.129165 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:47.130478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:47.132040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.133601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.134012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:47.135190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.135625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:47.135984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.135984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:47.135984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.135984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.135984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:47.135984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.135984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:47.139125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:47.139480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:47.139601 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:47.139800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:47.140266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:47.140586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.141290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:44:47.141290 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.141290 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:47.141290 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.141290 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.141290 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:47.141290 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.141290 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:47.142421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:47.142745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.142745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:47.142745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.142745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:47.142745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.142745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.142745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:47.142745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.142745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:47.143904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.143904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:47.143904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:47.143904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:47.143904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.143904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:47.143904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:47.143904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:47.143904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:47.143904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.143904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:47.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.144466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.144466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:47.144466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.144466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.144647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:47.145004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.145379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.145379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:47.145379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.145379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:47.175812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:47.176568 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:47.176767 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:47.177050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:47.177639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:47.178093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:47.178411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.179230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:47.179230 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.179230 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:47.179230 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.179230 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.179230 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:47.179230 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.179230 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.180365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:47.180756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.181082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:47.182128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:47.182648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:47.182648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:52.067603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:52.068392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:52.068686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:52.070111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:52.102890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:52.103017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.103899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:52.103899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.103899 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:52.103899 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.103899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.103899 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:52.103899 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.103899 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.104943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.105332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:52.105686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.105686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:52.106007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.106450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.106954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:52.106954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.106954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:52.119571 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:52.120759 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:52.120830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:52.121689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:52.121922 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:52.122054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.122609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:52.122609 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.122609 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:52.122609 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.122609 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.122609 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:52.122609 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.122609 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.123855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.124123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.125144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:52.125455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.125907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.125907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:52.125907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.125907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:52.126821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:52.127349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:52.127472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:52.127608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:52.128385 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:52.128554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.129058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:52.129058 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.129058 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:52.129058 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.129058 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.129058 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:52.129058 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.129058 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:52.130056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:52.130366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.130366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:52.130366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.130366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:52.130366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.130366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.130366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:52.130366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.130366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.131154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:52.131432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.131739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.131739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:52.131739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.131739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:52.132999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.133402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:52.133773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.133773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:52.133773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.133773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.133773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:52.133773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.133773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:52.140497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:52.141383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:52.141570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:52.141813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:52.142483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:52.142937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.144733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:44:52.144733 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.144733 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:52.144733 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.144733 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.144733 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:52.144733 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.144733 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:52.145790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:52.146309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.146309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:52.146309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.146309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:52.146309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.146309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.146309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:52.146309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.146309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:52.146767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.146767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:52.146767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:52.146767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:52.146767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.146767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:52.146767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:52.146767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:52.146767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:52.146767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.146767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:52.147115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:52.147572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:52.147970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.148526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:52.148526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:52.148526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.148526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:52.148526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.148526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:52.148526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.148526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.148526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:52.148526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.148526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:52.176591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:52.177274 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:52.177338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:52.177720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:52.178567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:52.178901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:52.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.179882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:52.179882 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.179882 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:52.179882 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.179882 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.179882 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:52.179882 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.179882 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.180818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:52.181116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.181412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:52.181556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:52.181899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:52.182300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:52.182300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:57.068159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:57.069203 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:57.069203 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:57.069720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:57.107148 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:57.107539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.108467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:44:57.108467 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.108467 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:57.108467 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.108467 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.108467 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:57.108467 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.108467 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.109429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.109864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:57.110397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:57.110857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.111396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:57.111396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.111396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.111396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:57.111396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.111396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:57.120822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:57.121246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:57.121394 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:57.122269 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:57.122489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:57.122659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.123349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:44:57.123349 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.123349 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:57.123349 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.123349 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.123349 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:57.123349 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.123349 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.124732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.125345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.125918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:57.126703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.127605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.127605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:57.127605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.127605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:57.127605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:57.127849 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:57.127927 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:57.128002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:44:57.128476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:57.128724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.131278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:44:57.131278 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.131278 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:57.131278 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.131278 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.131278 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:57.131278 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.131278 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:57.132396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:57.132950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.133419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.133893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:57.135413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.135988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:57.136535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.136535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:57.136535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.136535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.136535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:57.136535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.136535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:57.141270 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:57.141622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:57.141794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:57.141971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:57.142575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:57.142878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.143577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:44:57.143577 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.143577 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:57.143577 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.143577 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.143577 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:57.143577 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.143577 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:44:57.144350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:57.145135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:57.145798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.146463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:57.146463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:57.146463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.146463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:57.146463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.146463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:57.146463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.146463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.146463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:57.146463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.146463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.146942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:57.147785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.148781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.148781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:57.148781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.148781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:44:57.177536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:44:57.178111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:44:57.178262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:44:57.178465 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:44:57.179168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:57.179615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:44:57.179940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.180780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Traceback (most recent call last):
Apr 21 13:44:57.180780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.180780 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:44:57.180780 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.180780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.180780 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:44:57.180780 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.180780 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.181940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.182591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:44:57.183178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:44:57.183930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:44:57.184976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:44:57.184976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:44:57.184976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:44:57.184976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:44:57.184976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:44:57.184976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:02.069052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:02.070154 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:02.070154 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:02.070352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:02.109728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:02.110037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.110894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:02.110894 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.110894 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:02.110894 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.110894 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.110894 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:02.110894 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.110894 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.111980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.112391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:02.112778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.112778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:02.113103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.113576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.114299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:02.114299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.114299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:02.121272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:02.121780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:02.121867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:02.122917 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:02.123063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:02.123157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.123616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:02.123616 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.123616 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:02.123616 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.123616 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.123616 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:02.123616 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.123616 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.124926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.125358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.125810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:02.126814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.128113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.128113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:02.128113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.128113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:02.128113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:02.128879 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:02.129008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:02.129077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:02.129460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:02.129674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.130164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:45:02.130164 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.130164 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:02.130164 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.130164 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.130164 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:02.130164 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.130164 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:02.130796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:02.131485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.132142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.132729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:02.133302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.133987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:02.135519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.135519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:02.135519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.135519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.135519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:02.135519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.135519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:02.141713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:02.142487 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:02.142820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:02.143100 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:02.143504 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:02.143673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.144257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:45:02.144257 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.144257 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:02.144257 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.144257 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.144257 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:02.144257 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.144257 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:02.146106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:02.146672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.146672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:02.146672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.146672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:02.146672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.146672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.146672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:02.146672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.146672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:02.149954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.149954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:02.149954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:02.149954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:02.149954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.149954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:02.149954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:02.149954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:02.149954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:02.149954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.149954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:02.150324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.150804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.150804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:02.150804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.150804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.152334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:02.152763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.153107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.153107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:02.153107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.153107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:02.178285 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:02.178978 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:02.179084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:02.179338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:02.180410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:02.180971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:02.181260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.182356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Traceback (most recent call last):
Apr 21 13:45:02.182356 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.182356 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:02.182356 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.182356 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.182356 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:02.182356 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.182356 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.183755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.184635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:02.185529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.185529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:02.185747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:02.186688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:02.187667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:02.187667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:02.187667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:07.069636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:07.070375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:07.070375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:07.070933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:07.103614 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:07.103966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.104868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:07.104868 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.104868 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:07.104868 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.104868 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.104868 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:07.104868 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.104868 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.105739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.106169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:07.106559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:07.107000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.107591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:07.107591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.107591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.107591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:07.107591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.107591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:07.122084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:07.122619 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:07.122802 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:07.127184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:07.127646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:07.127811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.129092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:07.129092 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.129092 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:07.129092 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.129092 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.129092 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:07.129092 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.129092 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.131514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.132169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:07.132509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.132509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:07.132509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.133396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.134924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:07.135271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.135271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:07.135271 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:07.135271 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:07.135271 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:07.135271 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:07.135271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.135443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:45:07.135443 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.135443 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:07.135443 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.135443 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.135443 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:07.135443 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.135443 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:07.136221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:07.136586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.136586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:07.136586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.136586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:07.136586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.136586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.136586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:07.136586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.136586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.137050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:07.137749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:07.138044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.138321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.138678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:07.142356 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:07.142935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:07.143092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:07.143266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:07.143911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:07.144180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.147525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:45:07.147525 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.147525 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:07.147525 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.147525 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.147525 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:07.147525 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.147525 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:07.149628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:07.150066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:07.150375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.150634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:07.150634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:07.150634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.150634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:07.150634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.150634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:07.150634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.150634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.150634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:07.150634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.150634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.150880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:07.151206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.151586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.151586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:07.151586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.151586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:07.179242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:07.180679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:07.180679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:07.181028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:07.181853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:07.182362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:07.182574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.183574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Traceback (most recent call last):
Apr 21 13:45:07.183574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.183574 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:07.183574 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.183574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.183574 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:07.183574 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.183574 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.184373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.184980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:07.187181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:07.188449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:07.189024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:07.189024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:07.189024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:07.189024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:07.189024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:07.189024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:12.070424 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:12.071312 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:12.071312 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:12.071909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:12.096736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:12.097041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.097798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:12.097798 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.097798 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:12.097798 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.097798 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.097798 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:12.097798 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.097798 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.098688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.099069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:12.099439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:12.099826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.100429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:12.100429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.100429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.100429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:12.100429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.100429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:12.122653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:12.123276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:12.123377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:12.124323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:12.124462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:12.124632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.126734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:12.126734 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.126734 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:12.126734 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.126734 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.126734 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:12.126734 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.126734 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.127513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.128050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.128509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:12.129017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.129643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.129643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:12.129643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.129643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:12.132512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:12.132981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:12.133059 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:12.133174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:12.133639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:12.133862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.134315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:45:12.134315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.134315 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:12.134315 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.134315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.134315 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:12.134315 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.134315 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:12.135055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:12.135595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.135595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:12.135595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.135595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:12.135595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.135595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.135595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:12.135595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.135595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.135891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:12.136302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:12.136732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.137177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.137799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:12.142841 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:12.143338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:12.143465 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:12.143628 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:12.144554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:12.144788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.147335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:45:12.147335 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.147335 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:12.147335 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.147335 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.147335 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:12.147335 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.147335 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:12.148130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:12.148570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.148570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:12.148570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.148570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:12.148570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.148570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.148570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:12.148570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.148570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:12.148826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.148826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:12.148826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:12.148826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:12.148826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.148826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:12.148826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:12.148826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:12.148826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:12.148826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.148826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:12.149164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.149574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.149574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:12.149574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.149574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.149724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:12.150297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.150811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.150811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:12.150811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.150811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:12.180764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:12.181421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:12.181621 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:12.181943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:12.182476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Traceback (most recent call last):
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:12.186257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:12.186751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:12.187158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.187158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:12.187158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.187158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:12.187158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.187158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.187158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:12.187158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.187158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:12.187540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:12.188030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:12.188592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:12.188592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:12.188592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:17.071079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:17.071641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:17.074551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:17.074551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:17.104534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:17.104901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.106672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:17.106672 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.106672 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:17.106672 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.106672 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.106672 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:17.106672 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.106672 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.107771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.108237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:17.108678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.108678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:17.109323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.110416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.111491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:17.111491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.111491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:17.123483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:17.124840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:17.125020 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:17.126412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:17.126747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:17.126948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.127939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:45:17.127939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.127939 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:17.127939 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.127939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.127939 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:17.127939 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.127939 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.129046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.130168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:17.130925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.130925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:17.131040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.131494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.131947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:17.131947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.131947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:17.132981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:17.133249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:17.133334 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:17.133435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:17.134044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:17.134044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.134669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:17.134669 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.134669 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:17.134669 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.134669 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.134669 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:17.134669 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.134669 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:17.135607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.136007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:17.136007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.136007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:17.136007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.136007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.136007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:17.136007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.136007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.139576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:17.140006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.140568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.140568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:17.140568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.140568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.140742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:17.141423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.142122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.142122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:17.142122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.142122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:17.143496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:17.143805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:17.143983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:17.144177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:17.144709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:17.145005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.145636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:45:17.145636 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.145636 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:17.145636 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.145636 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.145636 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:17.145636 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.145636 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:17.146932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:17.147587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.147587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:17.147587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.147587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:17.147587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.147587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.147587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:17.147587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.147587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:17.148328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.148328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:17.148328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:17.148328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:17.148328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.148328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:17.148328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:17.148328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:17.148328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:17.148328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.148328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:17.149144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.149827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.149827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:17.149827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.149827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.150028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:17.150575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.151142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.151142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:17.151142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.151142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:17.182252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:17.182252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:17.182252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:17.182252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:17.182856 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:17.183150 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:17.183412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:17.186723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:17.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.187796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:17.188314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:17.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:17.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:17.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:17.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:17.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:17.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:17.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:17.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:17.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:17.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:22.071946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:22.072472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:22.072472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:22.072859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:22.104995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:22.105357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.106295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:22.106295 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.106295 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:22.106295 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.106295 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.106295 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:22.106295 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.106295 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.107436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.107876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:22.108233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.108233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:22.108536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.108942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.109380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:22.109380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.109380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:22.124884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:22.125315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:22.125532 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:22.126661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:22.126984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:22.127129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.127876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:45:22.127876 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.127876 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:22.127876 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.127876 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.127876 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:22.127876 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.127876 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.129414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.130685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:22.131286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:22.131954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.132817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:22.132817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.132817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.132817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:22.132817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.132817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:22.134336 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:22.134336 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:22.134336 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:22.134336 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:22.134719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:22.134719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.135107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:22.135107 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.135107 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:22.135107 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.135107 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.135107 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:22.135107 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.135107 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:22.135881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.136555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.136920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:22.137495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.138267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.138267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:22.138267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.138267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.138975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:22.139693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.140577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.140577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:22.140577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.140577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:22.144011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:22.144720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:22.144953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:22.145175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:22.145677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:22.145870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.146469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:45:22.146469 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.146469 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:22.146469 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.146469 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.146469 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:22.146469 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.146469 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:22.148078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:22.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:22.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:22.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:22.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:22.150414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.150414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:22.150414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:22.150414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:22.150414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.150414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:22.150414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:22.150414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:22.150414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:22.150414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.150414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:22.150752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.151167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.151167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:22.151167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.151167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:22.152830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.153357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.153357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:22.153357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.153357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:22.182332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:22.182887 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:22.183049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:22.183531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:22.184394 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:22.184686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:22.184823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.185638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:45:22.185638 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.185638 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:22.185638 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.185638 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.185638 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:22.185638 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.185638 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.187178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.187804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:22.188389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.188389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:22.188688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:22.189528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:22.190892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:22.190892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:22.190892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:27.073349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:27.073349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:27.073349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:27.074739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:27.112605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:27.113298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.114510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:27.114510 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.114510 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:27.114510 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.114510 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.114510 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:27.114510 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.114510 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.115991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.116738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:27.117408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:27.118169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.119115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:27.119115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.119115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.119115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:27.119115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.119115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:27.125438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:27.125767 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:27.125889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:27.127006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:27.127639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:27.128064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.129201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:45:27.129201 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.129201 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:27.129201 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.129201 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.129201 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:27.129201 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.129201 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.130505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.131147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:27.132173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.132173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:27.132470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.134248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.134789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:27.134789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.134789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:27.134789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:27.134789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:27.134789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:27.134789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:27.135541 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:27.135888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.136601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:27.136601 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.136601 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:27.136601 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.136601 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.136601 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:27.136601 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.136601 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:27.137505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.137960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:27.137960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.137960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:27.137960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.137960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.137960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:27.137960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.137960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.138382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:27.139026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.139892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:27.140554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.141132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:27.141132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:27.141132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.141132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:27.141132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.141132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:27.141132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.141132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.141132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:27.141132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.141132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:27.146797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:27.147670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:27.147801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:27.148011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:27.148694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:27.148864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.149352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:45:27.149352 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.149352 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:27.149352 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.149352 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.149352 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:27.149352 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.149352 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:27.150438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:27.150855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.150855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:27.150855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.150855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:27.150855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.150855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.150855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:27.150855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.150855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:27.151389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.151389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:27.151389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:27.151389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:27.151389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.151389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:27.151389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:27.151389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:27.151389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:27.151389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.151389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:27.151720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.152156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.152156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:27.152156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.152156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.152723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:27.153698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.154192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.154192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:27.154192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.154192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:27.182966 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:27.183758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:27.184137 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:27.185194 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:27.186199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:27.186711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:27.186972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.189117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:45:27.189117 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.189117 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:27.189117 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.189117 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.189117 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:27.189117 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.189117 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.192658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.193238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:27.193678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.193678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:27.195008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:27.195599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:27.196434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:27.196434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:27.196434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:32.073022 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:32.074081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:32.074081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:32.074081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:32.119787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:32.120086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.122694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:32.122694 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.122694 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:32.122694 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.122694 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.122694 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:32.122694 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.122694 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.123648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.124646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:32.125340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:32.126610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.128142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:32.128142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.128142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.128142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:32.128142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.128142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:32.128142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:32.128142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:32.128142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:32.128142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:32.128960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:32.128960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.128960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:45:32.128960 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.128960 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:32.128960 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.128960 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.128960 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:32.128960 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.128960 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.129512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.131460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:32.131782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:32.132074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.132433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:32.132433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.132433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.132433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:32.132433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.132433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:32.134572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:32.134809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:32.134857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:32.134933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:32.135305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:32.135367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.135661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:32.135661 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.135661 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:32.135661 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.135661 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.135661 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:32.135661 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.135661 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:32.136245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.136648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:32.136648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.136648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:32.136648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.136648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.136648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:32.136648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.136648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.136819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:32.137073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:32.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.137350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.137643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:32.138301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.138703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.138703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:32.138703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.138703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:32.147558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:32.148036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:32.148126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:32.148258 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:32.148704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:32.148960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.150095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:45:32.150095 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.150095 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:32.150095 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.150095 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.150095 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:32.150095 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.150095 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:32.150797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:32.151715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:32.152448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:32.153094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:32.154011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.154879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:32.183751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:32.185121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:32.185233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:32.185288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:32.185976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:32.186369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:32.186645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.187454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:32.187454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.187454 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:32.187454 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.187454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.187454 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:32.187454 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.187454 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.188151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:32.188850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:32.190368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:32.191769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:32.193246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:32.193246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:32.193246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:37.074459 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:37.075196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:37.075196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:37.075780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:37.110152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:37.110421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.111028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:37.111028 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.111028 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:37.111028 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.111028 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.111028 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:37.111028 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.111028 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.111833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.112239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:37.112587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:37.112991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.113504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:37.113504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.113504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.113504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:37.113504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.113504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:37.126421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:37.127082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:37.127320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:37.128429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:37.128689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:37.128854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.129685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:45:37.129685 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.129685 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:37.129685 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.129685 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.129685 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:37.129685 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.129685 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.131366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.131714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:37.131973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.131973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:37.133735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.134067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.134421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:37.134421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.134421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:37.135224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:37.135511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:37.135554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:37.135633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:37.135907 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:37.136061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.136425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:37.136425 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.136425 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:37.136425 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.136425 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.136425 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:37.136425 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.136425 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:37.136961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.137305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:37.137305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.137305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:37.137305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.137305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.137305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:37.137305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.137305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.137548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:37.137791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.138032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.138032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:37.138032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.138032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.141361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:37.141723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.142090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.142090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:37.142090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.142090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:37.148176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:37.148841 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:37.148933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:37.149067 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:37.149763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:37.150087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.151880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:45:37.151880 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.151880 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:37.151880 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.151880 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.151880 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:37.151880 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.151880 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:37.153402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:37.153826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.153826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:37.153826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.153826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:37.153826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.153826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.153826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:37.153826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.153826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:37.155140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.155140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:37.155140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:37.155140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:37.155140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.155140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:37.155140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:37.155140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:37.155140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:37.155140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.155140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:37.155520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.155874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.155874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:37.155874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.155874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.156588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:37.157062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.157972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.157972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:37.157972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.157972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:37.185248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:37.186042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:37.186042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:37.186263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:37.186865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:37.187197 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:37.187382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.188072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:37.188072 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.188072 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:37.188072 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.188072 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.188072 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:37.188072 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.188072 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.189360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:37.193565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:37.194341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:37.195821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:37.196614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:37.196614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:37.196614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:42.076314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:42.077073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:42.077073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:42.077470 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:42.080185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=434 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:45:42.108722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:42.108722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.109480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:42.109480 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.109480 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:42.109480 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.109480 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.109480 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:42.109480 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.109480 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.110472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.110860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:42.111249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:42.111696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.112276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:42.112276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.112276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.112276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:42.112276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.112276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:42.127094 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:42.128337 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:42.128337 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:42.128585 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:42.129055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:42.129438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.130759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:45:42.130759 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.130759 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:42.130759 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.130759 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.130759 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:42.130759 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.130759 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.131543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.131896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:42.132150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.132150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:42.132289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.132608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.132969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:42.132969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.132969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:42.135748 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:42.136086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:42.136141 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:42.136260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:42.136540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:42.136810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.137297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:42.137297 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.137297 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:42.137297 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.137297 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.137297 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:42.137297 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.137297 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:42.138140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.138517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:42.138517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.138517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:42.138517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.138517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.138517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:42.138517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.138517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.138977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:42.139443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.139755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.139755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:42.139755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.139755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.140278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:42.140818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.141330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.141330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:42.141330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.141330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:42.148708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:42.149762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:42.150289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:42.150289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:42.150750 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:42.151472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.152299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:45:42.152299 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.152299 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:42.152299 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.152299 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.152299 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:42.152299 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.152299 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:42.154251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:42.154755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:42.155221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:42.155619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:42.155983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.156598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:42.186993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:42.186993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:42.186993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:42.186993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:42.187343 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:42.187708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:42.187974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.188781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:42.188781 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.188781 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:42.188781 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.188781 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.188781 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:42.188781 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.188781 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.190219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:42.190627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.191065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:42.191715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:42.192175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:42.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:42.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:47.076670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:47.077473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:47.077473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:47.079407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:47.127747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:47.127747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.127747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:47.127747 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.127747 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:47.127747 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.127747 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.127747 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:47.127747 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.127747 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.128970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.129456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:47.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:47.129857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:47.130648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.131169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.131626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:47.131626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.131626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:47.131726 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:47.131838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:47.134720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:47.134720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:47.134720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.134842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:47.134842 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.134842 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:47.134842 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.134842 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.134842 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:47.134842 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.134842 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.135747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.136058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:47.136317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.136317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:47.137018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.137398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.137756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:47.137756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.137756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:47.137756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:47.137756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:47.137756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:47.137958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:47.140059 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:47.140059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.140059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:47.140059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.140059 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:47.140059 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.140059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.140059 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:47.140059 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.140059 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:47.140333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:47.140640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.140640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:47.140640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.140640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:47.140640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.140640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.140640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:47.140640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.140640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.141122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:47.141434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.141722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.141722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:47.141722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.141722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:47.142637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.142998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:47.143391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.143391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:47.143391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.143391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.143391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:47.143391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.143391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:47.149803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:47.150497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:47.150689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:47.151124 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:47.152590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:47.152590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.152700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:45:47.152700 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.152700 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:47.152700 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.152700 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.152700 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:47.152700 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.152700 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:47.154511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:47.154990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.154990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:47.154990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.154990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:47.154990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.154990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.154990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:47.154990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.154990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:47.157246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.157246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:47.157246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:47.157246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:47.157246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.157246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:47.157246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:47.157246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:47.157246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:47.157246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.157246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:47.157646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.158013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.158013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:47.158013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.158013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.158130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:47.158735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.159232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.159232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:47.159232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.159232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:47.186765 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:47.187489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:47.187775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:47.188156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:47.188823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:47.189702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:47.189702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.190008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:47.190008 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.190008 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:47.190008 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.190008 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.190008 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:47.190008 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.190008 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.191580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:47.191889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.192158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:47.193175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:47.193612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:47.193971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:47.193971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:52.077304 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:52.078303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:52.078303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:52.078542 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:52.109044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:52.109505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.110448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:45:52.110448 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.110448 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:52.110448 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.110448 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.110448 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:52.110448 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.110448 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.111917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.112967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:52.113656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:52.114479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.116982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:52.116982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.116982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.116982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:52.116982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.116982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:52.131774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:52.133526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:52.133695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:52.137071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:52.138867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:52.138867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.138867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:52.139003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:52.139071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:52.139207 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:52.139688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:52.140135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.141067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:52.141067 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.141067 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:52.141067 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.141067 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.141067 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:52.141067 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.141067 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:52.142868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.143337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:52.143337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.143337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:52.143337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.143337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.143337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:52.143337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.143337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.144558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:52.144920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.145497 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.146224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:52.146775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.147300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.147300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:52.147300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.147300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.148067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.148401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:52.148736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.148736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:52.149324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.149732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.150209 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:52.150209 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.150209 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:52.150316 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:52.151515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:52.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:52.151931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:52.152328 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:52.152929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.153103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:45:52.153103 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.153103 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:52.153103 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.153103 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.153103 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:52.153103 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.153103 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:52.153820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:52.154496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.154496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:52.154496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.154496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:52.154496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.154496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.154496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:52.154496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.154496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:52.155731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.155731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:52.155731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:52.155731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:52.155731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.155731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:52.155731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:52.155731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:52.155731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:52.155731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.155731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:52.156058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.156415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.156415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:52.156415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.156415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.156946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:52.157463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.158041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.158041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:52.158041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.158041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:52.188297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:52.188297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:52.188297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:52.188297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:52.188770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:52.189078 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:52.189235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.189825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:52.189825 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.189825 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:52.189825 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.189825 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.189825 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:52.189825 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.189825 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.190613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:52.191296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:52.191990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:52.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:52.193682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:52.193682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:52.193682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:57.077869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:57.079025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:57.079025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:57.079453 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:57.108370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:57.108758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.109645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:45:57.109645 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.109645 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:57.109645 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.109645 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.109645 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:57.109645 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.109645 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.110471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.111118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:57.111782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:57.112502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.113568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:57.113568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.113568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.113568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:57.113568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.113568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:57.134522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:57.135168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:57.135168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:57.136467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:57.136616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:57.136805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.137519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:57.137519 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.137519 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:57.137519 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.137519 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.137519 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:57.137519 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.137519 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.138438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.138783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:57.139130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.139130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.139900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.140391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:57.141069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.141069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:57.141069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:57.141069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:57.141069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:57.141069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:45:57.141467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:57.141648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.142168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:45:57.142168 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.142168 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:57.142168 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.142168 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.142168 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:57.142168 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.142168 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:57.143370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:57.143750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.143750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:57.143750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.143750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:57.143750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.143750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.143750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:57.143750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.143750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.145893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:57.146274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.146612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.146612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:57.146612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.146612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:57.147378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.147846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:57.148469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.148469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:57.148469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.148469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.148469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:57.148469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.148469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:57.150635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:57.152910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:57.152999 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:57.153156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:57.153578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:57.153840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.154414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:45:57.154414 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.154414 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:57.154414 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.154414 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.154414 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:57.154414 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.154414 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:45:57.155099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:57.155710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:57.156153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:57.156582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:45:57.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.157516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:45:57.188486 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:45:57.188747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:45:57.188837 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:45:57.189012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:45:57.189637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:57.189877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:45:57.190018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.190740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:45:57.190740 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.190740 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:45:57.190740 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.190740 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.190740 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:45:57.190740 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.190740 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.191935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:45:57.192385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.192795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:45:57.192996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:45:57.193547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:45:57.194030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:45:57.194030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:02.078695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:02.079350 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:02.079350 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:02.079984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:02.111634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:02.112640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.113546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:46:02.113546 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.113546 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:02.113546 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.113546 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.113546 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:02.113546 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.113546 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.116289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.116929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:02.117385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:02.117893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.118580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:02.118580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.118580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.118580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:02.118580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.118580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:02.134969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:02.135777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:02.136003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:02.137107 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:02.137444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:02.137684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.138863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:02.138863 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.138863 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:02.138863 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.138863 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.138863 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:02.138863 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.138863 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.140306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.140702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:02.141055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.141055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.142696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.143180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:02.143682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.143682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:02.143682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:02.143682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:02.143682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:02.143682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:02.143893 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:02.144191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.144774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:02.144774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.144774 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:02.144774 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.144774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.144774 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:02.144774 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.144774 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:02.148760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:02.149599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.150021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.150465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:02.150802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.151241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:02.151742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.151742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:02.151742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.151742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.151742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:02.151742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.151742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:02.152994 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:02.153360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:02.153498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:02.153655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:02.154232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:02.154849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.155642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:46:02.155642 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.155642 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:02.155642 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.155642 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.155642 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:02.155642 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.155642 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:02.156388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:02.156807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.156807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:02.156807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.156807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:02.156807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.156807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.156807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:02.156807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.156807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:02.157065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.157065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:02.157065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:02.157065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:02.157065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.157065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:02.157065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:02.157065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:02.157065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:02.157065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.157065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:02.157406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.157771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.157771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:02.157771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.157771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.157889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:02.158369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.158906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.158906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:02.158906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.158906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:02.189484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:02.189484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:02.189484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:02.189742 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:02.192252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:02.192252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:02.192252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.192252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:02.192252 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.192252 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:02.192252 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.192252 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.192252 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:02.192252 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.192252 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.193363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:02.193824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.194254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:02.194426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:02.194990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:02.195634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:02.195634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:07.079251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:07.079880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:07.082260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:07.082260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:07.120590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:07.120915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:07.124098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:07.124723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.125515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:07.126054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:07.126771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.126771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:07.126771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.126771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:07.126771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.126771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.126771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:07.126771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.126771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:07.135645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:07.136207 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:07.136314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:07.137558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:07.137720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:07.137836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.138355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:07.138355 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.138355 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:07.138355 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.138355 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.138355 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:07.138355 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.138355 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.139182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.139479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:07.139794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.139794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.140763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.141351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:07.141985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.141985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:07.143512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:07.144034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:07.144160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:07.144330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:07.144791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:07.144969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.148518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:07.148518 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.148518 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:07.148518 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.148518 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.148518 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:07.148518 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.148518 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:07.149514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:07.149998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.149998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:07.149998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.149998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:07.149998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.149998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.149998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:07.149998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.149998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.150384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:07.150792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.151187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.151187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:07.151187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.151187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:07.151447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.151952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:07.152462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.152462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:07.152462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.152462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.152462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:07.152462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.152462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:07.153373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:07.153945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:07.154143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:07.154401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:07.154919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:07.155135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.155884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:46:07.155884 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.155884 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:07.155884 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.155884 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.155884 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:07.155884 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.155884 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:07.157713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:07.158205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.158205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:07.158205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.158205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:07.158205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.158205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.158205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:07.158205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.158205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:07.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:07.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:07.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:07.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:07.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:07.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:07.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:07.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:07.159007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.159428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.159428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:07.159428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.159428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.160130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:07.160653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.161192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.161192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:07.161192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.161192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:07.189422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:07.189891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:07.190035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:07.190369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:07.190916 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:07.191129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:07.191304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.191916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:07.191916 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.191916 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:07.191916 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.191916 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.191916 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:07.191916 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.191916 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.193375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:07.194127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.194756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:07.194887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:07.195791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:07.196686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:07.196686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:12.079961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:12.080521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:12.080521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:12.081143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:12.115464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:12.115670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.116360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:46:12.116360 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.116360 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:12.116360 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.116360 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.116360 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:12.116360 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.116360 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.118803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.119274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:12.119665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:12.120142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.120720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:12.120720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.120720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.120720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:12.120720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.120720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:12.136293 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:12.136522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:12.136613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:12.137125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:12.137266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:12.137359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.137802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:12.137802 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.137802 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:12.137802 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.137802 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.137802 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:12.137802 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.137802 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.138451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.139023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.139594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:12.140334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.141194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.141194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:12.141194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.141194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:12.143916 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:12.144371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:12.144465 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:12.144566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:12.144934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:12.145137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.145639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:12.145639 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.145639 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:12.145639 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.145639 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.145639 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:12.145639 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.145639 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:12.146306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:12.146802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.146802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:12.146802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.146802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:12.146802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.146802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.146802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:12.146802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.146802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.147092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:12.147486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:12.147862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.148287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.148817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:12.153786 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:12.154709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:12.154925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:12.155175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:12.155872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:12.156446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.157371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:46:12.157371 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.157371 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:12.157371 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.157371 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.157371 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:12.157371 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.157371 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:12.158646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:12.159068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.159068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:12.159068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.159068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:12.159068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.159068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.159068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:12.159068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.159068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:12.159630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.159630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:12.159630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:12.159630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:12.159630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.159630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:12.159630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:12.159630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:12.159630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:12.159630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.159630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:12.160030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.160412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.160412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:12.160412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.160412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.161015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:12.162117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.162907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.162907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:12.162907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.162907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:12.190103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:12.190577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:12.190744 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:12.190982 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:12.191647 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:12.192177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:12.192521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.193454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:12.193454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.193454 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:12.193454 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.193454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.193454 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:12.193454 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.193454 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.194868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:12.195655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.196251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:12.196559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:12.197392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:12.198292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:12.198292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:17.080576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:17.081307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:17.081307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:17.081947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:17.126090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:17.126469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.127221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:46:17.127221 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.127221 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:17.127221 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.127221 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.127221 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:17.127221 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.127221 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.128125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.128648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:17.129172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.129172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:17.129342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.130310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.131670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:17.131670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.131670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:17.136803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:17.137878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:17.137878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:17.138501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:17.138834 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:17.139074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:17.142361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.143106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:17.143660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:17.144142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:17.144739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.144739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:17.144739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.144739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:17.144739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.144739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.144739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:17.144739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.144739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:17.144739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:17.145477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:17.145958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:17.146352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:17.147353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:17.147886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.148231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:17.148231 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.148231 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:17.148231 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.148231 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.148231 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:17.148231 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.148231 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:17.150036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:17.150663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.150663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:17.150663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.150663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:17.150663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.150663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.150663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:17.150663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.150663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.151613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:17.151907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.152184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.152184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:17.152184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.152184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:17.153198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.153602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:17.154016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.154016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:17.154016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.154016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.154016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:17.154016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.154016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:17.154355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:17.154785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:17.154876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:17.155012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:17.155509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:17.155785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.156335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:46:17.156335 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.156335 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:17.156335 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.156335 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.156335 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:17.156335 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.156335 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:17.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:17.157331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.157331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:17.157331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.157331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:17.157331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.157331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.157331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:17.157331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.157331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:17.157614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.157614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:17.157614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:17.157614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:17.157614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.157614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:17.157614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:17.157614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:17.157614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:17.157614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.157614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:17.157901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:17.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.158406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:17.158723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.159056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.159056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:17.159056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.159056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:17.190705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:17.191399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:17.191544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:17.191738 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:17.192500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:17.192856 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:17.193038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.193750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:17.193750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.193750 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:17.193750 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.193750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.193750 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:17.193750 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.193750 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:17.195355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:17.196014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:17.196886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:17.197805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:17.197805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:17.197805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:22.081540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:22.082976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:22.082976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:22.083975 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:22.127763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:22.128178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.129025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:46:22.129025 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.129025 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:22.129025 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.129025 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.129025 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:22.129025 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.129025 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.130316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.130971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:22.131511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:22.132097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.132915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:22.132915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.132915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.132915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:22.132915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.132915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:22.137536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:22.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:22.137950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:22.138682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:22.138885 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:22.139079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.139724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:22.139724 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.139724 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:22.139724 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.139724 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.139724 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:22.139724 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.139724 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.140867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.141234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:22.141584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.141584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.142098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.143726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:22.144342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.144342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:22.145519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:22.145853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:22.145981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:22.146115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:22.146542 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:22.146822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.147526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:22.147526 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.147526 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:22.147526 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.147526 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.147526 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:22.147526 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.147526 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:22.148354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:22.148794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.148794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:22.148794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.148794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:22.148794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.148794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.148794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:22.148794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.148794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.149639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:22.150336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:22.150703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.151082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.151617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:22.154827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:22.155695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:22.155968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:22.156228 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:22.156990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:22.157456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.161249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:22.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.162165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.162514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:22.162875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:22.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.163773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:22.163773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.163773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:22.163773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.163773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.163773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:22.163773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.163773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:22.191344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:22.191796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:22.191796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:22.191937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:22.192533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:22.192866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:22.193031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.193700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:22.193700 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.193700 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:22.193700 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.193700 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.193700 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:22.193700 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.193700 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.194592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:22.195456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:22.195990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:22.196701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:22.199661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:22.199661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:22.199661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:27.081876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:27.082745 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:27.083000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:27.084699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:27.127091 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:27.127091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.127449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:46:27.127449 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.127449 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:27.127449 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.127449 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.127449 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:27.127449 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.127449 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.128600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:27.129352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.129352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:27.129700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.130173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.130695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:27.130695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.130695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:27.138049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:27.138686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:27.138939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:27.140937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:27.140937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:27.140937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.141148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:27.141148 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.141148 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:27.141148 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.141148 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.141148 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:27.141148 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.141148 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.142715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.143085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:27.143539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.143539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.144450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.144957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:27.145452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.145452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:27.145869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:27.146838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:27.146964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:27.147105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:27.148184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:27.148184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.148574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:27.148574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.148574 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:27.148574 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.148574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.148574 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:27.148574 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.148574 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:27.149413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:27.149912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.149912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:27.149912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.149912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:27.149912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.149912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.149912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:27.149912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.149912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.150359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:27.150674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.150944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.150944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:27.150944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.150944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:27.153931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.154314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:27.154671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.154671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:27.154671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.154671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.154671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:27.154671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.154671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:27.155317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:27.155759 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:27.155865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:27.156096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:27.157093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:27.157093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.157405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:46:27.157405 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.157405 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:27.157405 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.157405 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.157405 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:27.157405 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.157405 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:27.158672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:27.159096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.159096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:27.159096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.159096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:27.159096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.159096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.159096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:27.159096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.159096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:27.159776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.159776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:27.159776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:27.159776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:27.159776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.159776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:27.159776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:27.159776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:27.159776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:27.159776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.159776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:27.160106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.160505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.160505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:27.160505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.160505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.161037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:27.161552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.162047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.162047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:27.162047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.162047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:27.191937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:27.193005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:27.193005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:27.193005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:27.193666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:27.196662 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:27.196662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.196973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:27.196973 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.196973 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:27.196973 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.196973 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.196973 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:27.196973 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.196973 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.197894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:27.198181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.198449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:27.198924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:27.199272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:27.199629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:27.199629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:32.082766 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:32.087234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:32.087234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:32.087234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:32.142773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:32.144669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:32.144669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:32.144669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:32.144669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:32.144933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.146920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:32.146920 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.146920 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:32.146920 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.146920 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.146920 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:32.146920 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.146920 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.151887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.152470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:32.152993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.152993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:32.152993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.155651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.156295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:32.157338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.157338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:32.157338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:32.157576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:32.157898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:32.158115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:32.160340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:32.160340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:32.160340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:32.161393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:32.161965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.163357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:32.163357 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.163357 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:32.163357 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.163357 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.163357 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:32.163357 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.163357 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:32.165381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:32.165904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.165904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:32.165904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.165904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:32.165904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.165904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.165904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:32.165904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.165904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.166392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:32.166758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.167128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.167128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:32.167128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.167128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:32.170188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.170702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:32.171169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.171169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:32.171169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.171169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.171169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:32.171169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.171169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:32.171582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:32.171831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.172654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:46:32.172654 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.172654 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:32.172654 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.172654 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.172654 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:32.172654 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.172654 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:32.175522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:32.175970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.175970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:32.175970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.175970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:32.175970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.175970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.175970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:32.175970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.175970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:32.182427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.015 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.182427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:32.182427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:32.182427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:32.182427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.182427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:32.182427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:32.182427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:32.182427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:32.182427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.182427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:32.182791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.183176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.183176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:32.183176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.183176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.185819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:32.187594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.188494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.188494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:32.188494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.188494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:32.193058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:32.193515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:32.194605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:32.194755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:32.195153 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:32.195566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.196668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:46:32.196668 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.196668 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:32.196668 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.196668 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.196668 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:32.196668 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.196668 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task check_fdb_aging_settings failed after 0.111 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.197721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.198720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:32.199559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.199559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:32.199949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.200815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.201381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:32.201381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.201381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:32.201381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:32.201381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:32.201381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:32.202388 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.202388 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:32.202388 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.202388 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.202388 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:32.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.203997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:32.204454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.204807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:32.206306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:32.206812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:32.207337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:32.207337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:37.085783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:37.085783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:37.086412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:37.086760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:37.117757 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.121344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.122596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:37.123585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.124574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:37.143371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:37.144082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:37.144426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:37.145381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:37.145739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:37.145916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.146591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:37.146591 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.146591 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:37.146591 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.146591 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.146591 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:37.146591 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.146591 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.147690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.147946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:37.148174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.148174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:37.149081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.149430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.149770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:37.149770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.149770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:37.154238 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:37.154584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:37.154753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:37.154962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:37.155425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:37.155740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.156287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:37.156287 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.156287 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:37.156287 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.156287 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.156287 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:37.156287 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.156287 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:37.157318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.157644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:37.157644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.157644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:37.157644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.157644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.157644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:37.157644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.157644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.158742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:37.158999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.159252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.159252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:37.159252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.159252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.160244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:37.160587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:37.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:37.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:37.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:37.161132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:37.161575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:37.163013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:37.163517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.164413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:46:37.164413 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.164413 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:37.164413 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.164413 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.164413 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:37.164413 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.164413 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:37.165656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:37.165944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.165944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:37.165944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.165944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:37.165944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.165944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.165944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:37.165944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.165944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:37.166903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.166903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:37.166903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:37.166903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:37.166903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.166903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:37.166903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:37.166903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:37.166903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:37.166903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.166903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:37.167122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.167372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.167372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:37.167372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.167372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.169249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:37.169706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.170027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.170027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:37.170027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.170027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:37.193632 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:37.194025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:37.194235 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:37.194411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:37.194908 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:37.195440 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:37.195766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.196745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:46:37.196745 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.196745 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:37.196745 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.196745 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.196745 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:37.196745 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.196745 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.197596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.198389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:37.198945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:37.199592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:37.201881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:37.201881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:37.201881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:37.201881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:37.201881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:37.201881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:42.085968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:42.089427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:42.089427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:42.089427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:42.090977 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:46:42.123359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:42.123814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:42.126343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:42.126970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.127442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:42.127847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:42.128450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.128450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:42.128450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.128450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:42.128450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.128450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.128450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:42.128450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.128450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:42.144377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:42.144822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:42.144940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:42.146162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:42.146451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:42.146618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.147385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:42.147385 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.147385 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:42.147385 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.147385 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.147385 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:42.147385 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.147385 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.148776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.149128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:42.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.149468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:42.150360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.151040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.151545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:42.151545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.151545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:42.154746 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:42.155112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:42.155254 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:42.155314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:42.156999 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:42.157186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.157622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:42.157622 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.157622 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:42.157622 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.157622 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.157622 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:42.157622 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.157622 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:42.158165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.158685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:42.158685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.158685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:42.158685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.158685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.158685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:42.158685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.158685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.158937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:42.159368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.159819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:42.160975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:42.161383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:42.161678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:42.161767 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:42.162431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:42.162820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.163672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:46:42.163672 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.163672 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:42.163672 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.163672 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.163672 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:42.163672 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.163672 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:42.164695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:42.165022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.165022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:42.165022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.165022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:42.165022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.165022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.165022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:42.165022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.165022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:42.165663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.165663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:42.165663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:42.165663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:42.165663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.165663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:42.165663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:42.165663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:42.165663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:42.165663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.165663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:42.165920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.166188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.166188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:42.166188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.166188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.166934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:42.167407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.167753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.167753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:42.167753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.167753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:42.194138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:42.200236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:42.200432 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:42.200702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:42.201195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:42.201604 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:42.201826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.203129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:46:42.203129 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.203129 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:42.203129 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.203129 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.203129 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:42.203129 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.203129 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.204347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.204615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:42.204853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.204853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:42.205419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:42.205722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:42.206145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:42.206145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:42.206145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:47.087538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:47.088589 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:47.088589 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:47.089058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:47.122856 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:47.123281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.124181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:46:47.124181 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.124181 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:47.124181 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.124181 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.124181 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:47.124181 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.124181 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.125199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.125687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:47.126103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.126103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:47.126293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.126794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.127341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:47.127341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.127341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:47.144892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:47.145348 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:47.145556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:47.146675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:47.147003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:47.147143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.148632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:47.148632 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.148632 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:47.148632 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.148632 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.148632 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:47.148632 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.148632 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.150100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.150720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:47.151517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.151517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:47.151999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.152449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.152909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:47.152909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.152909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:47.155182 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:47.155627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:47.155712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:47.155776 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:47.156376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:47.156569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.157106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:47.157106 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.157106 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:47.157106 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.157106 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.157106 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:47.157106 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.157106 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:47.157923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.158942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:47.159694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.160382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:47.161007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.162156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:47.163032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:47.163032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:47.163032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:47.163032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:47.163032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:47.163032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.163246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:46:47.163246 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.163246 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:47.163246 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.163246 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.163246 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:47.163246 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.163246 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:47.164351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:47.164795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.164795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:47.164795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.164795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:47.164795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.164795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.164795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:47.164795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.164795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:47.165712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.165712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:47.165712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:47.165712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:47.165712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.165712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:47.165712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:47.165712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:47.165712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:47.165712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.165712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:47.166115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.166562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.166562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:47.166562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.166562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.167436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:47.167941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.168433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.168433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:47.168433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.168433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:47.200274 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:47.203666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:47.203807 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:47.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:47.204345 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:47.204639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:47.204786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.205440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:46:47.205440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.205440 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:47.205440 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.205440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.205440 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:47.205440 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.205440 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.206292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.206938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:47.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:47.207847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:47.208426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:47.208426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:47.208426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:47.208426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:47.208426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:47.208426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:52.088268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:52.089645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:52.089645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:52.090353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:52.121170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:52.121412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.122245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:46:52.122245 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.122245 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:52.122245 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.122245 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.122245 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:52.122245 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.122245 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.123964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.125493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:52.126701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:52.128076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.129122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:52.129122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.129122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.129122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:52.129122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.129122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:52.145483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:52.145899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:52.146098 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:52.146721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:52.146885 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:52.147019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.147675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:52.147675 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.147675 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:52.147675 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.147675 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.147675 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:52.147675 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.147675 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.148395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.149006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:52.149610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:52.150397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.151411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:52.151411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.151411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.151411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:52.151411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.151411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:52.156247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:52.156880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:52.157523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:52.157523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:52.157523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:52.157698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.158234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:52.158234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.158234 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:52.158234 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.158234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.158234 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:52.158234 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.158234 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:52.159285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.159887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:52.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.160963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:52.161893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.162966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:52.163570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:52.163570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:52.163570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:52.163570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:52.163570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:52.163570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.163818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:46:52.163818 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.163818 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:52.163818 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.163818 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.163818 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:52.163818 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.163818 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:52.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:52.165047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.165047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:52.165047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.165047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:52.165047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.165047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.165047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:52.165047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.165047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:52.165395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.165395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:52.165395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:52.165395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:52.165395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.165395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:52.165395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:52.165395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:52.165395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:52.165395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.165395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:52.165681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.165959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.165959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:52.165959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.165959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.166230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:52.166624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:52.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:52.203746 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:52.206323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:52.206501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:52.207187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:52.207925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:52.208303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:52.208447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.209315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:46:52.209315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.209315 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:52.209315 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.209315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.209315 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:52.209315 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.209315 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.210684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.211373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:52.211971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.211971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:52.212324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:52.213113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:52.214123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:52.214123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:52.214123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:57.089154 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:57.089995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:57.089995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:57.091108 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:57.128201 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:57.128658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.129454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:46:57.129454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.129454 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:57.129454 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.129454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.129454 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:57.129454 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.129454 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.130129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.130766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:57.131393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:57.131853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.132385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:57.132385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.132385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.132385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:57.132385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.132385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:57.145925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:57.146389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:57.146389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:57.146820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:57.147112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:57.147337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.148274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:46:57.148274 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.148274 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:57.148274 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.148274 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.148274 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:57.148274 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.148274 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.148954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.149401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:57.149780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:57.150240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.150751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:57.150751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.150751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.150751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:57.150751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.150751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:57.158263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:57.158263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:57.158263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:57.158263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:46:57.158263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:57.158710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.159082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:46:57.159082 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.159082 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:57.159082 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.159082 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.159082 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:57.159082 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.159082 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:57.160001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.160373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:57.160373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.160373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:57.160373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.160373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.160373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:57.160373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.160373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.160822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:57.161075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.161391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.161391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:57.161391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.161391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.162150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:57.162532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.163083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.163083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:57.163083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.163083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:57.163083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:57.163083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:57.163083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:57.163083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:57.163337 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:57.163337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.164005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.164398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:46:57.164663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.164663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:57.164663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:57.164663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:57.164663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.164663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:57.164663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:57.164663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:57.164663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:57.164663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.164663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:57.164865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.165102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.165102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:57.165102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.165102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.165586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:57.165951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.166319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.166319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:57.166319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.166319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:46:57.206326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:46:57.206750 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:46:57.206826 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:46:57.207050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:46:57.207546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:57.207904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:46:57.208120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.208970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:46:57.208970 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.208970 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:46:57.208970 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.208970 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.208970 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:46:57.208970 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.208970 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.210475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.211124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:46:57.211686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.211686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:46:57.212824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:46:57.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:46:57.215163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:46:57.215163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:46:57.215163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:02.089897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:02.090603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:02.090712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:02.091914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:02.122512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:02.123002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.123989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:02.123989 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.123989 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:02.123989 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.123989 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.123989 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:02.123989 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.123989 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.124921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.125861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:02.126746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:02.127823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.129040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:02.129040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.129040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.129040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:02.129040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.129040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:02.146525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:02.146869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:02.146964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:02.147872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:02.148143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:02.148334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.149010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:02.149010 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.149010 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:02.149010 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.149010 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.149010 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:02.149010 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.149010 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.150134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.150719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:02.151225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.151225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:02.151322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.152121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.153036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:02.153036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.153036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:02.157202 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:02.157875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:02.157949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:02.158058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:02.158590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:02.158785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.160430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:02.160430 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.160430 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:02.160430 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.160430 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.160430 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:02.160430 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.160430 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:02.161298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.161826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:02.161826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.161826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:02.161826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.161826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.161826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:02.161826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.161826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.162069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:02.162451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.162779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.162779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:02.162779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.162779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.162994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:02.163447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.163828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.163828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:02.163828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.163828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:02.163828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:02.163828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:02.163828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:02.163828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:02.164173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:02.164428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.164937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:02.164937 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.164937 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:02.164937 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.164937 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.164937 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:02.164937 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.164937 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:02.165546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.166235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:02.166803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:02.167371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:02.167882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.168621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:02.206824 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:02.207641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:02.207816 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:02.207912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:02.208531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:02.208943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:02.209191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.210055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:47:02.210055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.210055 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:02.210055 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.210055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.210055 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:02.210055 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.210055 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.210830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.211181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:02.211511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.211511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:02.211574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:02.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:02.212450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:02.212450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:02.212450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:07.090530 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:07.091675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:07.091675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:07.092168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:07.136890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:07.136890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.137280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:07.137280 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.137280 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:07.137280 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.137280 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.137280 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:07.137280 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.137280 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.138024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.138881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:07.139579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:07.140452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.141047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:07.141047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.141047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.141047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:07.141047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.141047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:07.146872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:07.147145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:07.147262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:07.147753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:07.147930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:07.148080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:07.148500 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.148500 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:07.148500 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.148500 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.148500 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:07.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.148500 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.149977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.150390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:07.150810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:07.151308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.151870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:07.151870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.151870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.151870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:07.151870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.151870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:07.157675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:07.158083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:07.158223 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:07.158367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:07.158731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:07.158936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.159500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:47:07.159500 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.159500 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:07.159500 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.159500 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.159500 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:07.159500 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.159500 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:07.160503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:07.160856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.160856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:07.160856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.160856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:07.160856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.160856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.160856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:07.160856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.160856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:07.161673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.161943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.161943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:07.161943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.161943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:07.163231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.163587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:07.163938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.163938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:07.163938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.163938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.163938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:07.163938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.163938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:07.163938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:07.163938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:07.163938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:07.163938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:07.164238 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:07.164269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.166119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:07.166119 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.166119 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:07.166119 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.166119 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.166119 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:07.166119 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.166119 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:07.167014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.167398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:07.167398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.167398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:07.167398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.167398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.167398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:07.167398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.167398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:07.168056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.168438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.168713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:07.168713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.168713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.169546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:07.169995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.170410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.170410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:07.170410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.170410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:07.207720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:07.210573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:07.210573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:07.210573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:07.210573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:07.210573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:07.210573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.211021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:47:07.211021 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.211021 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:07.211021 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.211021 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.211021 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:07.211021 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.211021 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.213481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.213893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:07.214262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:07.214796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:07.215325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:07.215325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:07.215325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:07.215325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:07.215325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:07.215325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:12.091128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:12.092041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:12.092041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:12.092530 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:12.128544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:12.128902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.129716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:12.129716 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.129716 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:12.129716 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.129716 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.129716 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:12.129716 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.129716 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.131015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.131783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:12.132463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:12.133276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.134911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:12.134911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.134911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.134911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:12.134911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.134911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:12.148630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:12.149046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:12.149146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:12.149900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:12.150181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:12.150357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.151047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:12.151047 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.151047 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:12.151047 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.151047 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.151047 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:12.151047 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.151047 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.152048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.152599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:12.153136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:12.153816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.154643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:12.154643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.154643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.154643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:12.154643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.154643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:12.158082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:12.158375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:12.158483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:12.158605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:12.159003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:12.159204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.159782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:47:12.159782 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.159782 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:12.159782 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.159782 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.159782 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:12.159782 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.159782 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:12.160600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:12.161109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.161109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:12.161109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.161109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:12.161109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.161109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.161109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:12.161109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.161109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.161424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:12.161794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.162162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.162162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:12.162162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.162162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:12.162434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.162908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:12.163412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.163412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:12.163412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.163412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.163412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:12.163412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.163412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:12.163685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:12.164450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:12.164564 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:12.164700 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:12.165075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:12.165256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.165687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:12.165687 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.165687 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:12.165687 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.165687 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.165687 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:12.165687 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.165687 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:12.166800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.167242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:12.167242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.167242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:12.167242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.167242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.167242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:12.167242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.167242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:12.167853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.168220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:12.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.168988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:12.169492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.169996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.169996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:12.169996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.169996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:12.208495 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:12.209952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:12.210148 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:12.210489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:12.210950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:12.211228 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:12.211421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.212048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:47:12.212048 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.212048 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:12.212048 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.212048 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.212048 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:12.212048 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.212048 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.213025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.213766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:12.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:12.215600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:12.216165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:12.216165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:12.216165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:12.216165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:12.216165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:12.216165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:17.091706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:17.092821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:17.093026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:17.094178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:17.128972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:17.129621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.130234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:17.130234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.130234 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:17.130234 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.130234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.130234 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:17.130234 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.130234 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.131025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.131701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:17.132425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.132425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:17.132866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.133707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.134604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:17.134604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.134604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:17.149224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:17.149616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:17.149784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:17.150420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:17.150613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:17.150751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.151337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:17.151337 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.151337 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:17.151337 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.151337 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.151337 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:17.151337 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.151337 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.152260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.152804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:17.153331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:17.153910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.154467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:17.154467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.154467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.154467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:17.154467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.154467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:17.158665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:17.158935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:17.159041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:17.159112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:17.161690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:17.161939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.162581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:47:17.162581 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.162581 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:17.162581 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.162581 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.162581 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:17.162581 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.162581 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:17.163185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:17.163618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.163618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:17.163618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.163618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:17.163618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.163618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.163618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:17.163618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.163618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.163849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:17.164342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.164716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.164716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:17.164716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.164716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:17.165043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.165568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:17.166086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.166086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:17.166086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.166086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.166086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:17.166086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.166086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:17.166086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:17.166086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:17.166086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:17.166401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:17.166720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:17.166999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.167844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:17.167844 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.167844 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:17.167844 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.167844 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.167844 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:17.167844 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.167844 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:17.168901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.169344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:17.169344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.169344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:17.169344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.169344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.169344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:17.169344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.169344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:17.169845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.170232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.170579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:17.170579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.170579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.170963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:17.171418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.172012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.172012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:17.172012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.172012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:17.210001 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:17.210593 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:17.210723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:17.210950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:17.211423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:17.211714 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:17.211871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.212622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:47:17.212622 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.212622 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:17.212622 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.212622 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.212622 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:17.212622 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.212622 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.214019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.214405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:17.214744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.214744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:17.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:17.216467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:17.217019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:17.217019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:17.217019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:22.092935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:22.093823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:22.093823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:22.093951 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:22.124852 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:22.125105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.125694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:22.125694 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.125694 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:22.125694 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.125694 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.125694 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:22.125694 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.125694 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.126917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.128256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:22.129325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:22.129872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.130479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:22.130479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.130479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.130479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:22.130479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.130479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:22.149711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:22.150367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:22.151042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:22.151853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:22.152079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:22.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.152840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:22.152840 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.152840 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:22.152840 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.152840 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.152840 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:22.152840 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.152840 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.153991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.155260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:22.157336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:22.159006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.160923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:22.160923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.160923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.160923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:22.160923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.160923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:22.160923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:22.160923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:22.160923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:22.160923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:22.161252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:22.161252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.161560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:47:22.161560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.161560 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:22.161560 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.161560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.161560 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:22.161560 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.161560 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:22.162378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:22.163811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.164591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.165390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:22.166948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.167818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:22.168371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.168371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:22.168371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.168371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.168371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:22.168371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.168371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:22.168371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:22.168371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:22.168371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:22.168371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:22.169440 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:22.169440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.169440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:22.169440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.169440 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:22.169440 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.169440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.169440 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:22.169440 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.169440 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:22.170641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.171157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:22.171157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.171157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:22.171157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.171157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.171157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:22.171157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.171157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:22.172147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.172519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.172852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:22.172852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.172852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.177073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:22.177623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.178108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.178108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:22.178108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.178108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:22.210545 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:22.211393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:22.211627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:22.211944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:22.212529 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:22.212906 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:22.213149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.214432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:47:22.214432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.214432 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:22.214432 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.214432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.214432 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:22.214432 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.214432 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.215468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.215805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:22.216201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.216201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:22.216547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:22.216958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:22.217453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:22.217453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:22.217453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:27.093562 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:27.094276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:27.094276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:27.094940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:27.128165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:27.128527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.129333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:27.129333 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.129333 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:27.129333 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.129333 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.129333 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:27.129333 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.129333 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.130392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.130927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:27.131397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.131397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:27.131487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.132008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.132674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:27.132674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.132674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:27.150081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:27.150458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:27.151317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:27.151544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:27.152052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:27.152052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.152978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:27.152978 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.152978 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:27.152978 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.152978 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.152978 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:27.152978 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.152978 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.156246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.156966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:27.157451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:27.157919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.158785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:27.158785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.158785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.158785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:27.158785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.158785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:27.159461 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:27.160427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:27.160427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:27.160427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:27.160671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:27.160961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.162039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:47:27.162039 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.162039 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:27.162039 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.162039 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.162039 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:27.162039 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.162039 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:27.163063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:27.163765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.163765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:27.163765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.163765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:27.163765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.163765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.163765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:27.163765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.163765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.165543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:27.166175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:27.166753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.167288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:27.167851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:27.169299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:27.169299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:27.169299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:27.169299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:27.169568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:27.173240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.174057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.174763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:27.175361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.175926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.176774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:27.177602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.178417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.178417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:27.178417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.178417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:27.211146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:27.211616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:27.211721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:27.211920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:27.212399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:27.212732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:27.212946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.213767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:47:27.213767 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.213767 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:27.213767 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.213767 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.213767 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:27.213767 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.213767 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.215284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.215663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:27.216031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.216031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:27.216872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:27.217370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:27.217899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:27.217899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:27.217899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:32.094327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:32.095329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:32.095329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:32.095887 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:32.136927 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:32.137537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.139510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:32.139510 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.139510 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:32.139510 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.139510 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.139510 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:32.139510 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.139510 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.140292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.141272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:32.142494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:32.143549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.144678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:32.144678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.144678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.144678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:32.144678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.144678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:32.150613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:32.151060 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:32.151192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:32.151862 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:32.152092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:32.152324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.152906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:32.152906 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.152906 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:32.152906 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.152906 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.152906 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:32.152906 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.152906 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.154873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.155328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:32.155660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.155660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:32.156654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.157131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.158405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:32.158405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.158405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:32.159862 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:32.160540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:32.160654 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:32.160810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:32.161467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:32.161784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.162708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:47:32.162708 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.162708 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:32.162708 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.162708 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.162708 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:32.162708 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.162708 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:32.163724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:32.164153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.164153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:32.164153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.164153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:32.164153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.164153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.164153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:32.164153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.164153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.164595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:32.164925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.165443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.165443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:32.165443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.165443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:32.165719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.166158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:32.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:32.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:32.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:32.168309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:32.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:32.169448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:32.169534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:32.169950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:32.170183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.170785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:32.170785 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.170785 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:32.170785 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.170785 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.170785 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:32.170785 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.170785 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:32.172404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.172875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:32.172875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.172875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:32.172875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.172875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.172875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:32.172875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.172875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:32.174237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.174608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.174990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:32.174990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.174990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.176198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:32.176743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.178642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.178642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:32.178642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.178642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:32.211838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:32.212176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:32.212326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:32.212671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:32.213389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:32.213526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:32.213706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.214635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:47:32.214635 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.214635 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:32.214635 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.214635 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.214635 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:32.214635 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.214635 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.215690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.216338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:32.217043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.217043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:32.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:32.218666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:32.219571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:32.219571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:32.219571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:37.095084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:37.096180 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:37.096180 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:37.096388 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:37.134649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.135205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.135702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:37.136121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.136739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:37.151823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:37.152548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:37.152801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:37.154079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:37.154566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:37.154868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.155740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:37.155740 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.155740 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:37.155740 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.155740 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.155740 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:37.155740 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.155740 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.156700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.157470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:37.158036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:37.159028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.159779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:37.159779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.159779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.159779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:37.159779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.159779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:37.160287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:37.160835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:37.160967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:37.161104 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:37.161519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:37.161887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.162294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:47:37.162294 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.162294 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:37.162294 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.162294 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.162294 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:37.162294 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.162294 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:37.164689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:37.165242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.165242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:37.165242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.165242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:37.165242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.165242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.165242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:37.165242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.165242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.165986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:37.166404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.166796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.166796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:37.166796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.166796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:37.167159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.167637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:37.168153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.168153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:37.168153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.168153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.168153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:37.168153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.168153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:37.168828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:37.169387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:37.169521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:37.169696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:37.170261 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:37.170684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.171391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:37.171391 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.171391 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:37.171391 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.171391 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.171391 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:37.171391 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.171391 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:37.172966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.173634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:37.173634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.173634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:37.173634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.173634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.173634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:37.173634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.173634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:37.174498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.174930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.175332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:37.175332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.175332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.176409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:37.177038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.177633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.177633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:37.177633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.177633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:37.212450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:37.212983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:37.213089 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:37.213341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:37.213663 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:37.213931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:37.214146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.214815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:47:37.214815 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.214815 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:37.214815 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.214815 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.214815 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:37.214815 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.214815 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.215831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.216434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:37.217004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.217004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:37.217206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:37.217965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:37.218850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:37.218850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:37.218850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:42.095706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:42.096755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:42.096755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:42.097716 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:42.100147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:47:42.139956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:42.141103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.142155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:42.142155 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.142155 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:42.142155 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.142155 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.142155 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:42.142155 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.142155 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.143002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.143423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:42.143807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:42.144266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.145000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:42.145000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.145000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.145000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:42.145000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.145000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:42.152587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:42.152851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:42.152954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:42.153644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:42.153939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:42.154161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.154819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:42.154819 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.154819 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:42.154819 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.154819 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.154819 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:42.154819 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.154819 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.155718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.156077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:42.156524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.156524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:42.156798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.157275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.157744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:42.157744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.157744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:42.160679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:42.161517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:42.161724 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:42.161931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:42.162637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:42.162902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.163670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:47:42.163670 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.163670 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:42.163670 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.163670 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.163670 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:42.163670 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.163670 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:42.165219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:42.165810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.165810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:42.165810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.165810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:42.165810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.165810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.165810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:42.165810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.165810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.166597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:42.167140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.167692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.167692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:42.167692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.167692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:42.168223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.168740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:42.169316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.169316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:42.169316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.169316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.169316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:42.169316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.169316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:42.169734 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:42.170265 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:42.170412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:42.170592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:42.171173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:42.171707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.172408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:42.172408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.172408 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:42.172408 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.172408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.172408 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:42.172408 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.172408 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:42.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.174229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:42.174229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.174229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:42.174229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.174229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.174229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:42.174229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.174229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:42.175292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.175701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.176090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:42.176090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.176090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.177082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:42.177568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.178074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.178074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:42.178074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.178074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:42.212955 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:42.213635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:42.213817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:42.214098 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:42.214660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:42.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:42.215251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.216011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:47:42.216011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.216011 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:42.216011 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.216011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.216011 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:42.216011 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.216011 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.217944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.218910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:42.219689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:42.220536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:42.221502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:42.221502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:42.221502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:42.221502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:42.221502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:42.221502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:47.096489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:47.097863 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:47.097863 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:47.099084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:47.133307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:47.133670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.134404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:47.134404 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.134404 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:47.134404 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.134404 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.134404 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:47.134404 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.134404 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.135370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.136387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:47.137433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:47.139397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.140854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:47.140854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.140854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.140854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:47.140854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.140854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:47.153273 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:47.153839 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:47.154046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:47.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:47.155242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:47.155519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.156478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:47.156478 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.156478 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:47.156478 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.156478 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.156478 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:47.156478 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.156478 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.158688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.159167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:47.159945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.159945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:47.160542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.161131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.161968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:47.161968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.161968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:47.161968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:47.162203 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:47.162358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:47.162523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:47.163031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:47.163279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.163820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:47:47.163820 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.163820 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:47.163820 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.163820 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.163820 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:47.163820 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.163820 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:47.164967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:47.165764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.165764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:47.165764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.165764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:47.165764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.165764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.165764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:47.165764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.165764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.166407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:47.166975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:47.167611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.168278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.169051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:47.170225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:47.170814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:47.170911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:47.171103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:47.171705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:47.172055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.172889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:47.172889 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.172889 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:47.172889 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.172889 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.172889 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:47.172889 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.172889 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:47.173974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:47.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:47.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:47.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.174706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:47.175179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.175693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.176184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:47.176184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.176184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.176745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:47.177330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.177963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.177963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:47.177963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.177963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:47.213760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:47.214321 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:47.214520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:47.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:47.215308 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:47.215755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:47.216078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.217009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:47:47.217009 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.217009 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:47.217009 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.217009 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.217009 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:47.217009 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.217009 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.218418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.219136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:47.219847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.219847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:47.220034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:47.220892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:47.221896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:47.221896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:47.221896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:52.097407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:52.098259 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:52.098259 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:52.099662 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:52.135672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:52.135906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.136878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:52.136878 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.136878 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:52.136878 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.136878 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.136878 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:52.136878 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.136878 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.137754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.138402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:52.138971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:52.139689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.140625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:52.140625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.140625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.140625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:52.140625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.140625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:52.153935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:52.154441 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:52.154622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:52.155410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:52.155670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:52.156062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.156711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:52.156711 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.156711 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:52.156711 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.156711 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.156711 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:52.156711 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.156711 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.158004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.158620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:52.159156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.159156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:52.159620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.160485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.161186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:52.161186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.161186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:52.161932 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:52.162449 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:52.162652 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:52.162864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:52.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:52.163847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.164895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:47:52.164895 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.164895 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:52.164895 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.164895 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.164895 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:52.164895 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.164895 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:52.166609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:52.167045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.167045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:52.167045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.167045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:52.167045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.167045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.167045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:52.167045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.167045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.167976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:52.168409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.168792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.168792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:52.168792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.168792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:52.169537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.169967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:52.170513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.170513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:52.170513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.170513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.170513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:52.170513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.170513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:52.170711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:52.171371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:52.171371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:52.171371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:52.171802 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:52.172045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.172641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:52.172641 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.172641 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:52.172641 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.172641 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.172641 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:52.172641 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.172641 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:52.173726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.174680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:52.175446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:52.176382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:52.177115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.177660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:52.214362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:52.215914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:52.215914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:52.215914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:52.216569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:52.216963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:52.217243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.218248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:47:52.218248 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.218248 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:52.218248 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.218248 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.218248 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:52.218248 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.218248 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.221238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.221666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:52.222044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:52.222512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:52.223070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:52.223070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:52.223070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:52.223070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:52.223070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:52.223070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:57.097988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:57.098985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:57.098985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:57.099557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:57.131950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:57.132190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.132547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:47:57.132547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.132547 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:57.132547 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.132547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.132547 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:57.132547 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.132547 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.133482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.134071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:57.134726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:57.135461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.136955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:57.136955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.136955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.136955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:57.136955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.136955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:57.155382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:57.155382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:57.155382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:57.155985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:57.156250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:57.156474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.157329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:47:57.157329 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.157329 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:57.157329 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.157329 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.157329 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:57.157329 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.157329 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.158882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.159541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:57.159899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.159899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:57.160614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.161107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.162129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:57.162129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.162129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:57.162342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:57.163066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:57.163414 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:57.163687 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:47:57.164225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:57.164607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.165107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:47:57.165107 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.165107 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:57.165107 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.165107 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.165107 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:57.165107 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.165107 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:57.166588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:57.167068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.167068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:57.167068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.167068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:57.167068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.167068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.167068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:57.167068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.167068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.168007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:57.168394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.168767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.168767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:57.168767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.168767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:57.169825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.170321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:57.170845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.170845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:57.170845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.170845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.170845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:57.170845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.170845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:57.171185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:57.171785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:57.172013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:57.172293 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:57.172715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:57.172988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.173638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:47:57.173638 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.173638 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:57.173638 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.173638 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.173638 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:57.173638 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.173638 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:47:57.175058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.175520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:47:57.175520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.175520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:47:57.175520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.175520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.175520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:47:57.175520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.175520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:57.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.176941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.177307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:57.177307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.177307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:57.178353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.178867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.178867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:57.178867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.178867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:47:57.215372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:47:57.215847 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:47:57.216055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:47:57.216570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:47:57.217057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:57.217758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:47:57.217994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.219095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:47:57.219095 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.219095 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:47:57.219095 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.219095 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.219095 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:47:57.219095 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.219095 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.220889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.221284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:47:57.221661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.221661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:47:57.222632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:47:57.223091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:47:57.223650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:47:57.223650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:47:57.223650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:02.098635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:02.099400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:02.099768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:02.101096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:02.140899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:02.141158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.141850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:02.141850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.141850 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:02.141850 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.141850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.141850 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:02.141850 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.141850 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.142934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.143570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:02.144183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.144183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:02.144745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.145441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.147005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:02.147005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.147005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:02.155188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:02.155624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:02.155775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:02.157495 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:02.157880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:02.158152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.159105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:02.159105 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.159105 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:02.159105 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.159105 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.159105 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:02.159105 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.159105 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.160137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.160664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:02.161197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.161197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:02.161700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.162141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.162898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:02.162898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.162898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:02.162898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:02.163728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:02.163933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:02.164153 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:02.164803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:02.165147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.166083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:02.166083 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.166083 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:02.166083 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.166083 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.166083 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:02.166083 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.166083 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:02.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:02.168003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.168003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:02.168003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.168003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:02.168003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.168003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.168003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:02.168003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.168003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.168976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:02.169385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.169796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.169796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:02.169796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.169796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:02.170903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.171352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:02.171909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.171909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:02.171909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.171909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.171909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:02.171909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.171909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:02.171909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:02.171909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:02.172156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:02.172156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:02.172406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:02.172654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.173120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:02.173120 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.173120 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:02.173120 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.173120 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.173120 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:02.173120 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.173120 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:02.173937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.174490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:02.174490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.174490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:02.174490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.174490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.174490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:02.174490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.174490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:02.174803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.175138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.175507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:02.175507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.175507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.175917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:02.176425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.177190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.177190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:02.177190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.177190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:02.216104 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:02.216706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:02.216898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:02.217178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:02.217724 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:02.218013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:02.218240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.219055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:48:02.219055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.219055 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:02.219055 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.219055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.219055 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:02.219055 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.219055 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.221555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:02.222433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:02.223792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:02.225642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:02.225642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:02.225642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:02.225642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:02.225642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:02.225642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:07.099525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:07.100467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:07.100467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:07.101719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:07.138072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:07.138684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.139573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:07.139573 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.139573 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:07.139573 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.139573 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.139573 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:07.139573 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.139573 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.140743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.141508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:07.143110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:07.143885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.144954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:07.144954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.144954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.144954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:07.144954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.144954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:07.155857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:07.156401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:07.156744 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:07.157738 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:07.158049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:07.158327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.159352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:07.159352 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.159352 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:07.159352 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.159352 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.159352 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:07.159352 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.159352 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.160934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.161438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:07.161788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.161788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:07.162985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.163504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.164016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:07.164016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.164016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:07.164016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:07.164016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:07.164232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:07.164457 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:07.164811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:07.164997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.165547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:07.165547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.165547 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:07.165547 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.165547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.165547 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:07.165547 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.165547 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:07.166885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:07.167405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.167405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:07.167405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.167405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:07.167405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.167405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.167405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:07.167405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.167405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.167936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:07.168339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.168801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.168801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:07.168801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.168801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:07.169004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.169573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:07.170042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.170042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:07.170042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.170042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.170042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:07.170042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.170042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:07.172201 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:07.172560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:07.172629 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:07.172734 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:07.174140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:07.174368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.174899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:07.174899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.174899 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:07.174899 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.174899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.174899 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:07.174899 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.174899 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:07.175902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.176415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:07.176415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.176415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:07.176415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.176415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.176415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:07.176415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.176415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:07.177518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.177915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.178303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:07.178303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.178303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.179322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:07.179861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.180409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.180409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:07.180409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.180409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:07.216809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:07.217759 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:07.217759 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:07.217985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:07.218530 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:07.218879 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:07.219112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.219989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:48:07.219989 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.219989 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:07.219989 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.219989 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.219989 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:07.219989 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.219989 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.221436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.222143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:07.222954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:07.223786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:07.224791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:07.224791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:07.224791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:07.224791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:07.224791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:07.224791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:12.100497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:12.101381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:12.101381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:12.103611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:12.137477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:12.137764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.138407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:12.138407 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.138407 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:12.138407 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.138407 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.138407 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:12.138407 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.138407 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.139326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.139749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:12.140188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.140188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:12.140319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.140807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.141415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:12.141415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.141415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:12.156483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:12.157038 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:12.157255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:12.158159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:12.158449 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:12.158647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.159444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:12.159444 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.159444 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:12.159444 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.159444 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.159444 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:12.159444 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.159444 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.161172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.161833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:12.162491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.162491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:12.162956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.163734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.164200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:12.164200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.164200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:12.164200 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:12.165689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:12.165817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:12.165984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:12.166836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:12.167112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.167900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:12.167900 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.167900 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:12.167900 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.167900 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.167900 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:12.167900 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.167900 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:12.168878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:12.169494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.169494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:12.169494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.169494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:12.169494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.169494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.169494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:12.169494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.169494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.169792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:12.170141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.170509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.170509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:12.170509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.170509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:12.170825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.171304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:12.171844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.171844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:12.171844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.171844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.171844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:12.171844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.171844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:12.172774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:12.173045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:12.173140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:12.173246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:12.173624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:12.173915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.174441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:12.174441 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.174441 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:12.174441 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.174441 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.174441 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:12.174441 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.174441 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:12.175165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.176011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:12.176011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.176011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:12.176011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.176011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.176011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:12.176011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.176011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:12.176475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.177118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:12.178188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:12.179012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.180003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:12.180003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.180003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:12.180003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.180003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.180003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:12.180003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.180003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:12.217715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:12.218388 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:12.219019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:12.219019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:12.219558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:12.219852 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:12.220135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.221004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:48:12.221004 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.221004 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:12.221004 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.221004 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.221004 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:12.221004 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.221004 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.222836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.223487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:12.224075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.224075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:12.224630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:12.225192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:12.225918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:12.225918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:12.225918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:17.101409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:17.102661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:17.102979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:17.104379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:17.141882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:17.141882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.143422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:17.143422 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.143422 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:17.143422 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.143422 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.143422 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:17.143422 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.143422 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.145244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.145909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:17.146633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.146633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:17.147135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.147669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.148270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:17.148270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.148270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:17.157176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:17.157708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:17.157909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:17.159166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:17.159709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:17.159932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.160957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:17.160957 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.160957 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:17.160957 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.160957 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.160957 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:17.160957 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.160957 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.162958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.163736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:17.164305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.164305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:17.164877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.165982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.167305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:17.167305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.167305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:17.167305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:17.167305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:17.167305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:17.167305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:17.167305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:17.167782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.168013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:17.168013 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.168013 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:17.168013 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.168013 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.168013 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:17.168013 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.168013 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:17.168941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:17.169870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.169870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:17.169870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.169870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:17.169870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.169870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.169870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:17.169870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.169870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.170400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:17.171049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:17.171833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.172557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:17.173514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:17.174421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:17.174421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:17.174539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:17.174807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.175422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:17.175422 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.175422 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:17.175422 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.175422 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.175422 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:17.175422 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.175422 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:17.176490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.176839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:17.176839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.176839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:17.176839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.176839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.176839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:17.176839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.176839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:17.177384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.177830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.178088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:17.178088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.178088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.178537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:17.178886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.179313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.179313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:17.179313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.179313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:17.218466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:17.219108 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:17.219389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:17.219706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:17.220568 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:17.221561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.222589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:48:17.222589 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.222589 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:17.222589 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.222589 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.222589 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:17.222589 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.222589 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.224101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.224756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:17.225345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.225345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:17.225690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:17.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:17.227106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:17.227106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:17.227106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:22.102197 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:22.103084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:22.103084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:22.104417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:22.140386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:22.141017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.142227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:22.142227 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.142227 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:22.142227 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.142227 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.142227 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:22.142227 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.142227 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.143098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.143869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:22.144712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:22.145531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.146908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:22.146908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.146908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.146908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:22.146908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.146908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:22.157825 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:22.158553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:22.158553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:22.159348 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:22.159861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:22.160142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.161920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:22.161920 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.161920 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:22.161920 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.161920 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.161920 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:22.161920 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.161920 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.163527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.164175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:22.164839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.164839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:22.166386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.166860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.167443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:22.167443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.167443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:22.167443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:22.167443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:22.167443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:22.167703 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:22.168327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:22.168710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.169452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:22.169452 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.169452 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:22.169452 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.169452 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.169452 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:22.169452 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.169452 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:22.170892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:22.171397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.171397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:22.171397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.171397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:22.171397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.171397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.171397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:22.171397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.171397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.172581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:22.172964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.173332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.173332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:22.173332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.173332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:22.174343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.174818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:22.175352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.175352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:22.175352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.175352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.175352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:22.175352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.175352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:22.175352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:22.175352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:22.175352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:22.175352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:22.175968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:22.176491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.177407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:22.177407 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.177407 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:22.177407 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.177407 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.177407 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:22.177407 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.177407 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:22.179018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.179567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:22.179567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.179567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:22.179567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.179567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.179567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:22.179567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.179567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:22.181406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.181812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.182239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:22.182239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.182239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.182807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:22.183307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.183866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.183866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:22.183866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.183866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:22.219142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:22.219912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:22.220105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:22.220412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:22.221598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:22.222069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:22.222425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.223508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:48:22.223508 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.223508 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:22.223508 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.223508 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.223508 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:22.223508 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.223508 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.224901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.225594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:22.226485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:22.227246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:22.228203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:22.228203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:22.228203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:22.228203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:22.228203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:22.228203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:27.102968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:27.104011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:27.104011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:27.105103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:27.141765 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:27.142023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.142748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:27.142748 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.142748 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:27.142748 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.142748 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.142748 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:27.142748 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.142748 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.143827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.144723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:27.145645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:27.146755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.148027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:27.148027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.148027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.148027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:27.148027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.148027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:27.158441 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:27.159053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:27.159332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:27.160393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:27.161089 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:27.161412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.162481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:27.162481 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.162481 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:27.162481 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.162481 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.162481 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:27.162481 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.162481 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.165042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.165871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:27.166650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:27.167627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.168503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:27.168503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.168503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.168503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:27.168503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.168503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:27.168503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:27.168503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:27.168503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:27.168503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:27.168986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:27.168986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.169590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:27.169590 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.169590 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:27.169590 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.169590 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.169590 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:27.169590 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.169590 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:27.170952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:27.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:27.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:27.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:27.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.171742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.172157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:27.172726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.173324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.173324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:27.173324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.173324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:27.173625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.174353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:27.175151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.175151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:27.175151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.175151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.175151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:27.175151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.175151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:27.175494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:27.175847 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:27.175985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:27.176143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:27.176843 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:27.177367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.178197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:27.178197 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.178197 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:27.178197 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.178197 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.178197 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:27.178197 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.178197 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:27.179512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.180083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:27.180083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.180083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:27.180083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.180083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.180083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:27.180083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.180083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:27.180887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.181685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.182361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:27.182361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.182361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.182680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:27.183387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.184243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.184243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:27.184243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.184243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:27.219862 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:27.220519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:27.220707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:27.220995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:27.221733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:27.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:27.222414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.223426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:48:27.223426 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.223426 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:27.223426 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.223426 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.223426 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:27.223426 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.223426 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.225027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.225953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:27.227096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:27.228159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:27.229362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:27.229362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:27.229362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:27.229362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:27.229362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:27.229362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:32.103686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:32.104725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:32.104725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:32.105956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:32.140023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:32.140471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.141466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:32.141466 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.141466 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:32.141466 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.141466 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.141466 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:32.141466 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.141466 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.142262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.143142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:32.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:32.144746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.146061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:32.146061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.146061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.146061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:32.146061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.146061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:32.159055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:32.159749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:32.159894 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:32.161039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:32.161039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:32.161269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.162045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:32.162045 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.162045 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:32.162045 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.162045 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.162045 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:32.162045 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.162045 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.162811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.163541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:32.164104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:32.164576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.165094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:32.165094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.165094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.165094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:32.165094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.165094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:32.168117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:32.168979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:32.169071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:32.169154 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:32.169489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:32.169685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.170393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:32.170393 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.170393 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:32.170393 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.170393 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.170393 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:32.170393 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.170393 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:32.171673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:32.172551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.173001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.173418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:32.173818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.174324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:32.174938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.174938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:32.174938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.174938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.174938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:32.174938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.174938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:32.175849 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:32.176411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:32.176411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:32.176889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:32.177317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:32.177792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.178658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:32.178658 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.178658 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:32.178658 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.178658 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.178658 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:32.178658 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.178658 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:32.179777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.180614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:32.181044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:32.181483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.181483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:32.181483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.181483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:32.181483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.181483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.181483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:32.181483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.181483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.182066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:32.182592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.183461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.183461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:32.183461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.183461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:32.220583 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:32.221721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:32.221721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:32.221721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:32.222228 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:32.222532 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:32.222683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.224325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:48:32.224325 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.224325 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:32.224325 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.224325 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.224325 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:32.224325 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.224325 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.224925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.225435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:32.225867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:32.226420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:32.227051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:32.227051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:32.227051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:32.227051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:32.227051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:32.227051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:37.104506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:37.105386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:37.105386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:37.106118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:37.139354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:37.139593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.140257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:37.140257 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.140257 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:37.140257 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.140257 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.140257 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:37.140257 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.140257 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.141050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.142596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:37.143366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:37.144156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.145336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:37.145336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.145336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.145336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:37.145336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.145336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:37.159681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:37.162195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:37.162195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:37.162195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:37.162195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:37.162477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.164949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:37.164949 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.164949 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:37.164949 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.164949 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.164949 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:37.164949 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.164949 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.165856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.167506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:37.167870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.167870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:37.168357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.168820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.169468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:37.169468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.169468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:37.169468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:37.169644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:37.169786 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:37.169912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:37.170544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:37.170849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.171565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:37.171565 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.171565 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:37.171565 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.171565 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.171565 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:37.171565 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.171565 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:37.172204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:37.172670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.172670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:37.172670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.172670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:37.172670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.172670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.172670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:37.172670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.172670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.172931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:37.173313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:37.173730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.174105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.174535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:37.176065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:37.176331 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:37.176412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:37.176516 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:37.176838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:37.177074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.177580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:37.177580 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.177580 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:37.177580 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.177580 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.177580 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:37.177580 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.177580 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:37.178408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.178734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:37.178734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.178734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:37.178734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.178734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.178734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:37.178734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.178734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:37.179264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.179524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.179793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:37.179793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.179793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:37.180808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.181169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.181169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:37.181169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.181169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:37.221322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:37.221895 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:37.222015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:37.222231 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:37.222820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:37.223245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:37.223523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.224387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:48:37.224387 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.224387 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:37.224387 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.224387 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.224387 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:37.224387 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.224387 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.225464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.226061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:37.226691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.226691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:37.226893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:37.227783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:37.228729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:37.228729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:37.228729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:42.105845 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:42.106601 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:42.106830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:42.108264 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:42.110558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:48:42.143272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:42.143491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.144054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:42.144054 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.144054 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:42.144054 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.144054 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.144054 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:42.144054 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.144054 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.144929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.145427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:42.145855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:42.146343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.146878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:42.146878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.146878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.146878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:42.146878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.146878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:42.160691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:42.160993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:42.161160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:42.162046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:42.162256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:42.162463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.163147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:42.163147 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.163147 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:42.163147 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.163147 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.163147 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:42.163147 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.163147 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.164094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.164722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:42.165340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:42.166170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.167099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:42.167099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.167099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.167099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:42.167099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.167099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:42.169726 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:42.170505 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:42.170737 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:42.170931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:42.171354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:42.171608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.172808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:42.172808 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.172808 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:42.172808 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.172808 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.172808 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:42.172808 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.172808 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:42.173669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:42.174308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.174308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:42.174308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.174308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:42.174308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.174308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.174308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:42.174308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.174308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.174692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:42.175092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.176000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.176000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:42.176000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.176000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:42.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.177659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:42.178138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.178138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:42.178138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.178138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.178138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:42.178138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.178138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:42.178138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:42.178138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:42.178439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:42.178572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:42.178990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:42.179251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.179705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:42.179705 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.179705 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:42.179705 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.179705 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.179705 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:42.179705 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.179705 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:42.181310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.181787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:42.181787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.181787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:42.181787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.181787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.181787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:42.181787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.181787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:42.182182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.182601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.182974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:42.182974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.182974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.183639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:42.184427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.185105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.185105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:42.185105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.185105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:42.221888 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:42.222367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:42.222502 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:42.222691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:42.223453 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:42.223983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:42.224362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.226176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:48:42.226176 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.226176 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:42.226176 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.226176 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.226176 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:42.226176 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.226176 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.227120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.227711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:42.228402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.228402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:42.228705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:42.229398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:42.229887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:42.229887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:42.229887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:47.107561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:47.108501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:47.108718 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:47.110015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:47.148874 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:47.149269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.150146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:47.150146 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.150146 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:47.150146 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.150146 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.150146 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:47.150146 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.150146 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.151194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.151779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:47.152290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:47.152889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.153624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:47.153624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.153624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.153624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:47.153624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.153624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:47.160985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:47.161788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:47.161987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:47.162882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:47.163098 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:47.163305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.163964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:47.163964 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.163964 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:47.163964 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.163964 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.163964 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:47.163964 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.163964 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.165649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.166308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:47.166930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.166930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:47.167909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.168823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.169728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:47.169728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.169728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:47.170311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:47.170945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:47.171167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:47.171412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:47.171828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:47.172031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.172623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:47.172623 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.172623 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:47.172623 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.172623 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.172623 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:47.172623 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.172623 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:47.173935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:47.174388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.174388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:47.174388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.174388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:47.174388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.174388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.174388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:47.174388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.174388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.175369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:47.175711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.176088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.176088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:47.176088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.176088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:47.176502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.176913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:47.177385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.177385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:47.177385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.177385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.177385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:47.177385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.177385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:47.177980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:47.178479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:47.178617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:47.178769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:47.179400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:47.179652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.180115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:47.180115 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.180115 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:47.180115 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.180115 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.180115 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:47.180115 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.180115 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:47.180999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.181499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:47.181906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.182381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.182757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:47.182757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.182757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.183041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:47.183654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.184168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.184168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:47.184168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.184168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:47.222417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:47.222891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:47.223044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:47.223369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:47.224153 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:47.224458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:47.224634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.225307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:48:47.225307 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.225307 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:47.225307 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.225307 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.225307 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:47.225307 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.225307 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.226667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.227393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:47.227964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.227964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:47.228470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:47.229168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:47.230151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:47.230151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:47.230151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:52.108398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:52.109307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:52.109307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:52.110648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:52.143194 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:52.143603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.144491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:52.144491 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.144491 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:52.144491 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.144491 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.144491 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:52.144491 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.144491 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.145446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.146139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:52.146860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:52.147677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.148855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:52.148855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.148855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.148855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:52.148855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.148855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:52.161906 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:52.162598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:52.162749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:52.163858 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:52.164233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:52.164484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.165509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:52.165509 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.165509 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:52.165509 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.165509 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.165509 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:52.165509 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.165509 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.167228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.167570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:52.168244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.168244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:52.171408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.172473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.173780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:52.173780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.173780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:52.173780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:52.173780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:52.174194 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:52.174331 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:52.174823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:52.175017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.175695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:52.175695 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.175695 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:52.175695 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.175695 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.175695 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:52.175695 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.175695 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:52.177128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:52.177823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.177823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:52.177823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.177823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:52.177823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.177823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.177823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:52.177823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.177823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.179958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:52.180368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.180708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.180708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:52.180708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.180708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:52.181221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.181587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:52.181991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.181991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:52.181991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.181991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.181991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:52.181991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.181991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:52.181991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:52.181991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:52.181991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:52.181991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:52.182562 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:52.182811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.183352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:52.183352 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.183352 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:52.183352 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.183352 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.183352 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:52.183352 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.183352 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:52.184127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.184502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:52.184502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.184502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:52.184502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.184502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.184502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:52.184502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.184502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:52.184951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.185188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.185500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:52.185500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.185500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.186072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:52.186568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.187069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.187069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:52.187069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.187069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:52.222969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:52.223972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:52.223972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:52.223972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:52.224721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:52.225059 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:52.225277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.226083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:48:52.226083 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.226083 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:52.226083 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.226083 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.226083 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:52.226083 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.226083 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.227160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.227661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:52.228136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.228136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:52.228538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:52.229066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:52.229627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:52.229627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:52.229627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:57.109061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:57.109889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:57.109996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:57.111365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:57.151539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:57.151960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.153043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:48:57.153043 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.153043 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:57.153043 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.153043 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.153043 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:57.153043 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.153043 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.154200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.155082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:57.155770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:57.156692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.157675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:57.157675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.157675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.157675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:57.157675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.157675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:57.162623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:57.163103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:57.163103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:57.163764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:57.163983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:57.164163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.164745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:48:57.164745 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.164745 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:57.164745 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.164745 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.164745 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:57.164745 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.164745 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.166187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.166603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:57.167010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.167010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:57.168522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.169101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.169637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:57.169637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.169637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:57.172541 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:57.173462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:57.174827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:57.175041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:48:57.175738 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:57.176085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:48:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.177870 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:57.179357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:57.181062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.181062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:57.181062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.181062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:57.181062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.181062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.181062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:57.181062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.181062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.181584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:57.182740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:57.183455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.184260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:57.185125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.185817 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:48:57.186394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.187098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:57.188390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:57.189052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:57.189335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.189704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:48:57.223725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:48:57.224514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:48:57.224761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:48:57.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:48:57.225978 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:57.226601 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:48:57.226675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.227809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:48:57.227809 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.227809 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:48:57.227809 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.227809 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.227809 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:48:57.227809 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.227809 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.230967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:48:57.231825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.231825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:48:57.233926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:48:57.234630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:48:57.235156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:48:57.235156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:48:57.235156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:02.109755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:02.110655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:02.110655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:02.111263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:02.142566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:02.142728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.143341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:49:02.143341 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.143341 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:02.143341 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.143341 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.143341 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:02.143341 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.143341 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.144233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.144819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:02.145439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:02.146122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.147051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:02.147051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.147051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.147051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:02.147051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.147051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:02.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:02.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:02.164644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:02.165341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:02.166099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:02.166099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.167087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:49:02.167087 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.167087 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:02.167087 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.167087 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.167087 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:02.167087 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.167087 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.168617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.170165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:02.171089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:02.171533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.171934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:02.171934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.171934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.171934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:02.171934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.171934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:02.173034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:02.173292 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:02.173351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:02.173437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:02.173659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:02.173805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.174189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:02.174189 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.174189 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:02.174189 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.174189 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.174189 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:02.174189 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.174189 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:02.174733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:02.175277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.175277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:02.175277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.175277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:02.175277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.175277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.175277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:02.175277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.175277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.175568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:02.175844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:02.176115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.176476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.177041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:02.183385 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:02.183863 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:02.184010 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:02.184176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:02.184580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:02.184813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.185412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:49:02.185412 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.185412 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:02.185412 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.185412 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.185412 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:02.185412 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.185412 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:02.186785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:02.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:02.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:02.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.187376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:02.188138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.188570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.188951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:02.188951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.188951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.189901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:02.190614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.191190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.191190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:02.191190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.191190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:02.224308 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:02.225769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:02.225769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:02.225769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:02.226357 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:02.226769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:02.227004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.227900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:49:02.227900 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.227900 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:02.227900 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.227900 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.227900 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:02.227900 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.227900 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.230511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:02.231394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.231394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:02.231706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:02.232531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:02.233339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:02.233339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:02.233339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:07.110507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:07.111540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:07.111540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:07.112648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:07.150529 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:07.150830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.152018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:49:07.152018 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.152018 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:07.152018 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.152018 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.152018 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:07.152018 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.152018 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.152701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.153881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:07.154954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:07.156157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.157660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:07.157660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.157660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.157660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:07.157660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.157660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:07.163783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:07.164360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:07.164555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:07.165428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:07.165803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:07.165999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.166696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:49:07.166696 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.166696 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:07.166696 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.166696 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.166696 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:07.166696 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.166696 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.168323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.168885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:07.169292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.169292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:07.170094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.170564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.171070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:07.171070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.171070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:07.173526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:07.174322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:07.174460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:07.174629 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:07.175283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:07.175626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.176793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:07.176793 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.176793 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:07.176793 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.176793 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.176793 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:07.176793 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.176793 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:07.178140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:07.178768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.178768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:07.178768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.178768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:07.178768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.178768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.178768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:07.178768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.178768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.179486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:07.179904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.180358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.180358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:07.180358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.180358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:07.181398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.182058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:07.182823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.182823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:07.182823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.182823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.182823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:07.182823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.182823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:07.183861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:07.184244 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:07.184407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:07.184552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:07.185041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:07.185523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.186294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:49:07.186294 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.186294 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:07.186294 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.186294 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.186294 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:07.186294 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.186294 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:07.187435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.188012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:07.188012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.188012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:07.188012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.188012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.188012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:07.188012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.188012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:07.188746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.189187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.189594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:07.189594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.189594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.189904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:07.190690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.191292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.191292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:07.191292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.191292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:07.225198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:07.226103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:07.226368 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:07.226720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:07.227421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:07.227905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:07.228158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.229174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:49:07.229174 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.229174 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:07.229174 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.229174 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.229174 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:07.229174 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.229174 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.230451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.231006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:07.231629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.231629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:07.232600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:07.233383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:07.234195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:07.234195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:07.234195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:12.111255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:12.112515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:12.112515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:12.113408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:12.149911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:12.150291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.151173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:49:12.151173 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.151173 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:12.151173 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.151173 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.151173 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:12.151173 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.151173 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.152012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.152704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:12.153412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:12.154376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.154943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:12.154943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.154943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.154943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:12.154943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.154943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:12.164308 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:12.164897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:12.165066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:12.165980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:12.166329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:12.166582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.167408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:49:12.167408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.167408 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:12.167408 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.167408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.167408 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:12.167408 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.167408 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.169240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.169990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:12.170567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.170567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:12.171602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.172391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.173355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:12.173355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.173355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:12.174084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:12.174730 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:12.174958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:12.175202 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:12.175889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:12.176292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.177190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:12.177190 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.177190 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:12.177190 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.177190 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.177190 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:12.177190 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.177190 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:12.178870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:12.179414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.179414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:12.179414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.179414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:12.179414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.179414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.179414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:12.179414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.179414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.180457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:12.180862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.181356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.181356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:12.181356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.181356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:12.181867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.182371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:12.182891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.182891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:12.182891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.182891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.182891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:12.182891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.182891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:12.184453 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:12.185056 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:12.185267 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:12.185486 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:12.186045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:12.186609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.187392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:49:12.187392 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.187392 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:12.187392 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.187392 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.187392 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:12.187392 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.187392 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:12.188602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.189066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:12.189066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.189066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:12.189066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.189066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.189066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:12.189066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.189066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:12.189820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.190222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.190866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:12.190866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.190866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.191540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:12.192167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.192813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.192813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:12.192813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.192813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:12.225721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:12.226029 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:12.226140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:12.226361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:12.226725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:12.227033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:12.227225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.227922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:49:12.227922 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.227922 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:12.227922 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.227922 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.227922 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:12.227922 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.227922 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.228744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.229492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:12.230175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:12.230968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:12.232093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:12.232093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:12.232093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:12.232093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:12.232093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:12.232093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:17.112036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:17.112808 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:17.113011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:17.114426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:17.158811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:17.159182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.160091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:49:17.160091 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.160091 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:17.160091 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.160091 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.160091 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:17.160091 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.160091 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.160880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.161165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:17.161465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.161465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:17.161689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.162034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.162436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:17.162436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.162436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:17.164841 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:17.165721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:17.165939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:17.166922 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:17.167325 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:17.167601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.168424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:49:17.168424 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.168424 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:17.168424 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.168424 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.168424 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:17.168424 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.168424 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.170051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.170326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:17.170573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.170573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:17.171940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.172271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.172638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:17.172638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.172638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:17.174517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:17.175383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:17.175581 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:17.175816 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:17.176357 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:17.176711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.177148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:17.177148 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.177148 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:17.177148 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.177148 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.177148 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:17.177148 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.177148 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:17.178761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:17.179102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.179102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:17.179102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.179102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:17.179102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.179102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.179102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:17.179102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.179102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.180471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:17.180849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.181095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.181095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:17.181095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.181095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:17.182513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.182852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:17.183195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.183195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:17.183195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.183195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.183195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:17.183195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.183195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:17.185003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:17.185592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:17.185800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:17.186028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:17.186793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:17.187315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.188194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:49:17.188194 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.188194 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:17.188194 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.188194 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.188194 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:17.188194 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.188194 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:17.189663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.189984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:17.189984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.189984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:17.189984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.189984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.189984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:17.189984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.189984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:17.191437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.191698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.191949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:17.191949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.191949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:17.194191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.194579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.194579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:17.194579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.194579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:17.226444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:17.227023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:17.227147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:17.227406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:17.227990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:17.228602 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:17.228862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.229890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:49:17.229890 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.229890 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:17.229890 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.229890 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.229890 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:17.229890 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.229890 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.230859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.231470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:17.232009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:17.232744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:17.233662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:17.233662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:17.233662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:17.233662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:17.233662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:17.233662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:22.112684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:22.113395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:22.113395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:22.113998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:22.145675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:22.146054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.146924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:49:22.146924 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.146924 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:22.146924 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.146924 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.146924 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:22.146924 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.146924 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.147853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:22.149651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:22.150451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.151399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:22.151399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.151399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.151399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:22.151399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.151399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:22.165595 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:22.167110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:22.167110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:22.167511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:22.167812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:22.168049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.169395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:49:22.169395 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.169395 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:22.169395 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.169395 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.169395 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:22.169395 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.169395 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.170483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.171067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:22.171652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.171652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:22.172522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.173446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.174287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:22.174287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.174287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:22.175121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:22.175561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:22.176023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:22.176023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:22.176478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:22.176740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.177489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:22.177489 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.177489 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:22.177489 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.177489 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.177489 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:22.177489 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.177489 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:22.178823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:22.179319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.179319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:22.179319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.179319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:22.179319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.179319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.179319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:22.179319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.179319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.180163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:22.180613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.180990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.180990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:22.180990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.180990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:22.181804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.182301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:22.182818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.182818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:22.182818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.182818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.182818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:22.182818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.182818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:22.185495 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:22.186008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:22.186362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:22.186362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:22.186865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:22.187229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.187940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:49:22.187940 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.187940 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:22.187940 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.187940 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.187940 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:22.187940 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.187940 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:22.189429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.189885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:22.189885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.189885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:22.189885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.189885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.189885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:22.189885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.189885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:22.191237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.191944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.192668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:22.192668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.192668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.193018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:22.193936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.194882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.194882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:22.194882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.194882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:22.227021 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:22.228026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:22.228242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:22.228606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:22.229172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:22.229594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:22.229837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.230759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:49:22.230759 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.230759 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:22.230759 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.230759 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.230759 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:22.230759 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.230759 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.232569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.232963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:22.233322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.233322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:22.233657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:22.234108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:22.234650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:22.234650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:22.234650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:27.113338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:27.114052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:27.114052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:27.115399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:27.151705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:27.151909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.152542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:49:27.152542 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.152542 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:27.152542 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.152542 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.152542 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:27.152542 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.152542 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.153632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.154349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:27.155180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:27.155952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.156894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:27.156894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.156894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.156894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:27.156894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.156894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:27.166220 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:27.166807 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:27.166945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:27.167846 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:27.168166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:27.168447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.169199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:49:27.169199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.169199 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:27.169199 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.169199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.169199 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:27.169199 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.169199 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.170337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.170672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:27.171123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.171123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:27.172868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.173980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.174807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:27.174807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.174807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:27.175602 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:27.176112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:27.176370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:27.176653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:27.177379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:27.177509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.178309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:27.178309 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.178309 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:27.178309 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.178309 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.178309 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:27.178309 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.178309 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:27.179724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:27.180302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.180302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:27.180302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.180302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:27.180302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.180302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.180302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:27.180302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.180302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.181161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:27.181577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.181950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.181950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:27.181950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.181950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:27.182498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.182933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:27.183468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.183468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:27.183468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.183468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.183468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:27.183468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.183468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:27.185969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:27.187004 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:27.187281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:27.187389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:27.187945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:27.188181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.188775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:49:27.188775 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.188775 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:27.188775 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.188775 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.188775 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:27.188775 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.188775 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:27.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.192551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:27.192551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.192551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:27.192551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.192551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.192551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:27.192551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.192551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:27.193821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.194186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.194603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:27.194603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.194603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.196015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:27.196555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.197058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.197058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:27.197058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.197058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:27.227846 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:27.228803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:27.229045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:27.229423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:27.230029 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:27.230492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:27.230729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.231795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:49:27.231795 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.231795 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:27.231795 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.231795 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.231795 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:27.231795 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.231795 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.233544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.234123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:27.234781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.234781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:27.236500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:27.237151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:27.237701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:27.237701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:27.237701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:32.113995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:32.140714 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:32.141002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:32.142569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:32.168907 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:32.169731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:32.171256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:32.176111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:32.177873 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:32.178127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.179640 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:32.180284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:49:32.180284 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.180284 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:32.180284 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.180284 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.180284 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:32.180284 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.180284 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:32.183249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.186582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.187464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:32.188160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.188160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:32.188160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:32.191138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.192016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.192968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:32.192968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.192968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:32.192968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:32.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:32.196317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:32.196423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:32.196583 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:32.200076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:32.200076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.200929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:32.200929 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.200929 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:32.200929 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.200929 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.200929 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:32.200929 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.200929 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:32.202151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:32.202720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.202720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:32.202720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.202720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:32.202720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.202720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.202720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:32.202720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.202720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.015 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.203018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:32.203531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.203968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.203968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:32.203968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.203968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:32.204234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.204731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:32.205401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.205401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:32.205401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.205401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.205401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:32.205401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.205401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:32.205401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:32.205401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.208938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:49:32.208938 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.208938 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:32.208938 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.208938 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.208938 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:32.208938 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.208938 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:32.213266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:32.213772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.213772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:32.213772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.213772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:32.213772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.213772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.213772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:32.213772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.213772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:32.218306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.017 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.218306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:32.218306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:32.218306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:32.218306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.218306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:32.218306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:32.218306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:32.218306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:32.218306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.218306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:32.218720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.219117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.219117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:32.219117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.219117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.222622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:32.224206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.225175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.225175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:32.225175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.225175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:32.225175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:32.225175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.226064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:49:32.226064 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.226064 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:32.226064 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.226064 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.226064 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:32.226064 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.226064 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task check_fdb_aging_settings failed after 0.085 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.227183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.227729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:32.228098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.228098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:32.228543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.229087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.229588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:32.229588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.229588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:32.229588 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:32.229854 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:32.230030 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:32.230348 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:32.230707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:32.230883 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:32.231011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.231668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:49:32.231668 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.231668 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:32.231668 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.231668 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.231668 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:32.231668 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.231668 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.232940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.233318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:32.233680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.233680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:32.234079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:32.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:32.235054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:32.235054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:32.235054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:37.140672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:37.141537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:37.142055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:37.173491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e4595cde-982b-4563-af1c-5c8650c333f7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:37.174655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:37.176427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:37.176576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:37.181143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:37.181458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:37.181563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.181929 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:37.182590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:37.183897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:37.184661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:37.184661 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.184661 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:37.184661 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.184661 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.184661 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:37.184661 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.184661 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:37.185200 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.188179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.188807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:37.189614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.189614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:37.190869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.191476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:37.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:37.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:37.192068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.192262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:49:37.192262 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.192262 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:37.192262 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.192262 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.192262 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:37.192262 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.192262 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:37.194110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:37.194991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.194991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:37.194991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.194991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:37.194991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.194991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.194991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:37.194991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.194991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.200159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:37.200658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.201046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.201046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:37.201046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.201046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:37.201046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:37.204724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.205608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:37.206533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.206533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:37.206533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.206533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.206533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:37.206533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.206533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:37.206533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:37.206533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:37.206533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:37.208148 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:37.208578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.212258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.213522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:37.217807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.217807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:37.217807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:37.217807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:37.217807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.217807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:37.217807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:37.217807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:37.217807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:37.217807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.217807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:37.218657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.219151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.219151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:37.219151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.219151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.221474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:37.222160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.222648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.222648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:37.222648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.222648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:37.229608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:37.230059 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:37.230221 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:37.230463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:37.233689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:37.233931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:37.234085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.236733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:49:37.236733 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.236733 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:37.236733 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.236733 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.236733 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:37.236733 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.236733 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.243241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:37.243752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.244080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.245183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.245636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:37.246059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.246059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:37.248958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:37.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.250033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e4595cde-982b-4563-af1c-5c8650c333f7 None None] Traceback (most recent call last):
Apr 21 13:49:37.250033 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.250033 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:37.250033 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.250033 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.250033 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:37.250033 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.250033 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e4595cde-982b-4563-af1c-5c8650c333f7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.108 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.251607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.251975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:37.252347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.252347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e4595cde-982b-4563-af1c-5c8650c333f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:37.253482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:37.254045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:37.254546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:37.254546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:37.254546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:42.141307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:42.142194 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:42.142327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:42.143743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:42.146446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:49:42.176358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:42.177017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:42.178011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:42.179282 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:42.179548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:42.179717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.180586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:49:42.180586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.180586 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:42.180586 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.180586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.180586 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:42.180586 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.180586 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.182225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.182551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:42.182833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.182833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:42.184832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:42.188236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.189254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.190444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:42.190444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.190444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:42.190444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:42.190444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:42.190444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:42.190444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.190444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:42.190887 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:42.190887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.191057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:42.191057 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.191057 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:42.191057 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.191057 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.191057 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:42.191057 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.191057 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.192013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.192743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:42.193405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:42.194205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:42.195102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.196184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:42.196810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:42.197366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.197777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.198440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.199395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:42.202325 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:42.202580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:42.202655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:42.202775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:42.203235 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:42.203709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.204422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:49:42.204422 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.204422 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:42.204422 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.204422 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.204422 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:42.204422 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.204422 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:42.205405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.205886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:42.205886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.205886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:42.205886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.205886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.205886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:42.205886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.205886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:42.206378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.206790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.207196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:42.207196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.207196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:42.208387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.209111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.209111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:42.209111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.209111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:42.230397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:42.230867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:42.231049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:42.231397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:42.232066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:42.232502 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:42.232750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.233675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:49:42.233675 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.233675 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:42.233675 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.233675 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.233675 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:42.233675 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.233675 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.235528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.236323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:42.236985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.236985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:42.238649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:42.239538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:42.240546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:42.240546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:42.240546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:46.460500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:46.461558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:46.461558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:46.476356 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 21 13:49:46.476356 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106267) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 21 13:49:46.476684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Fixing resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:49:46.518385 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, resource={'id': 'b57c058f-054e-4a2a-980f-6bfda2111e90', 'name': 'private', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['6a3993e6-b5ba-4812-baa3-b25db5880cd9', 'c97deb27-a1d6-4e03-b489-95abde8fd791'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:29:32Z', 'updated_at': '2026-04-21T13:29:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 24380}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.518906 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.519492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3f6c78fa-b671-412d-bb34-5426a40d8bff, external_ids_update=None, columns={'tag': 24380}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.519696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.520887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Traceback (most recent call last):
Apr 21 13:49:46.520887 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.520887 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:46.520887 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.520887 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.520887 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:46.520887 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.520887 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:46.522332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Failed to fix resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.522332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:46.522332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:49:46.522332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:49:46.522332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:49:46.522332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:49:46.522332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:49:46.522332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:46.522332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.522332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.522332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.522931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.523540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:46.523540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.523540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:46.523540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Fixing resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:49:46.555877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d0607710-681d-444d-be26-dbeca5a07b46, resource={'id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'name': 'public', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-21T13:29:38Z', 'updated_at': '2026-04-21T13:29:48Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.556370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d0607710-681d-444d-be26-dbeca5a07b46, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.557020 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-19f04b40-ed99-482a-8774-a76892a716d1, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.557420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.558413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Traceback (most recent call last):
Apr 21 13:49:46.558413 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.558413 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:46.558413 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.558413 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.558413 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:46.558413 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.558413 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:46.559848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Failed to fix resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.559848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:46.559848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:49:46.559848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:49:46.559848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:49:46.559848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:49:46.559848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:49:46.559848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:46.559848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.559848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.559848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.560408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.560997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:46.560997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.560997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:46.560997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Fixing resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:49:46.589390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, resource={'id': '345e31fe-ede8-4dd0-823e-e81dcbc5bef0', 'name': 'admin_net', 'tenant_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'project_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['bb271c4b-4172-418f-995c-f72abd439dbe'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:34:46Z', 'updated_at': '2026-04-21T13:34:47Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 22486}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.589751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.590319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-c1b4e94e-49ea-4c1f-8edf-2f1433fdff16, external_ids_update=None, columns={'tag': 22486}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.590502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.591311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Traceback (most recent call last):
Apr 21 13:49:46.591311 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.591311 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:46.591311 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.591311 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.591311 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:46.591311 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.591311 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:46.592568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Failed to fix resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.592568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:46.592568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:49:46.592568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:49:46.592568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:49:46.592568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:49:46.592568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:49:46.592568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:46.592568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.592568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.592568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.593167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.593826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:46.593826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.593826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:46.593826 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Fixing resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:49:46.618978 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, resource={'id': 'b364ad09-f70d-45d8-b9dd-8420f5f1c377', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['620e9559-2c15-4370-904f-f4677bc69542'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:35:24Z', 'updated_at': '2026-04-21T13:35:25Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 61006}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.619404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.619904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a100d4e8-7f11-40c9-8e4a-346c2c764f69, external_ids_update=None, columns={'tag': 61006}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.620107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.620923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Traceback (most recent call last):
Apr 21 13:49:46.620923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.620923 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:46.620923 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.620923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.620923 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:46.620923 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.620923 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:46.622093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Failed to fix resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.622093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:46.622093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:49:46.622093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:49:46.622093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:49:46.622093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:49:46.622093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:49:46.622093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:46.622093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.622093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.622093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.622703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.623403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:46.623403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.623403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:46.623403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Fixing resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:49:46.696977 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, resource={'id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'enable_snat': True}, 'gw_port_id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-61m1f', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'ip_address': '172.24.5.49', 'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'}, {'ip_address': '2001:db8::162', 'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 3, 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04'}, resource_type=routers, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.697387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.697727 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.697940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.698120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.699103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Traceback (most recent call last):
Apr 21 13:49:46.699103 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.699103 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:46.699103 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.699103 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.699103 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:46.699103 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.699103 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:46.699357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Unable to update router 06899d4c-a436-45e4-bed5-95dabe888f51. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.700513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Failed to fix resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.700513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:46.700513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:49:46.700513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:49:46.700513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:49:46.700513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:49:46.700513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 21 13:49:46.700513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 21 13:49:46.700513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 21 13:49:46.700513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 21 13:49:46.700513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:46.700860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:46.701330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:46.701330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:46.701330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.701330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:46.701330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.701330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.701330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:46.701330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.701330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:46.701330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Fixing resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:49:46.807330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, resource={'id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'name': '', 'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'project_id': '', 'mac_address': 'fa:16:3e:fb:71:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000168688', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.807853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3 ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'neutron:network_name': 'neutron-d0607710-681d-444d-be26-dbeca5a07b46', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-06899d4c-a436-45e4-bed5-95dabe888f51'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.49/24', '2001:db8::162/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.808393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, lrouter_port=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:46.808865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.809895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Traceback (most recent call last):
Apr 21 13:49:46.809895 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.809895 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:46.809895 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.809895 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.809895 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:46.809895 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.809895 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:46.811149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] Maintenance task: Failed to fix resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.811149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:46.811149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:49:46.811149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:49:46.811149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:49:46.811149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:49:46.811149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 21 13:49:46.811149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:46.811149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.811149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.811149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:46.811662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:46.812019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:46.812019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:46.812019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:46.812019 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-381e4f82-311a-4fd1-a1e3-7198199ef848 None None] OVN maintenance task check_for_inconsistencies finished in 0.350 seconds
Apr 21 13:49:47.142083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:47.142684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:47.142878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:47.144151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:47.177202 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:47.178134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:47.179471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:47.181477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:47.181821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:47.183365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.184907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:47.184907 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.184907 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:47.184907 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.184907 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.184907 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:47.184907 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.184907 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.186552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.186914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:47.187287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.187287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:47.188506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.189046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.189611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:47.189611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.189611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:47.189611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:47.189611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.189832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:47.190297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:49:47.190297 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.190297 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:47.190297 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.190297 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.190297 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:47.190297 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.190297 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:47.190717 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.191858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.192240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:47.192616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.192616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:47.192616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:47.193617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.194082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.194675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:47.194675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.194675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:47.194675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:47.194675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:47.194907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.195912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:49:47.195912 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.195912 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:47.195912 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.195912 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.195912 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:47.195912 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.195912 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:47.197622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:47.198028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.198028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:47.198028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.198028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:47.198028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.198028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.198028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:47.198028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.198028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.199476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:47.199846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.200255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.200255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:47.200255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.200255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:47.201390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.201786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:47.202278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.202278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:47.202278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.202278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.202278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:47.202278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.202278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:47.202707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:47.203286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:47.203286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:47.203286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:47.203643 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:47.204038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.205335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:49:47.205335 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.205335 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:47.205335 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.205335 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.205335 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:47.205335 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.205335 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:47.205896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.206712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:47.207422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:47.208069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:47.209407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.211664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:47.230953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:47.231544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:47.231760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:47.232049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:47.232619 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:47.233065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:47.233398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.234230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:49:47.234230 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.234230 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:47.234230 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.234230 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.234230 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:47.234230 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.234230 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.236019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.236634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:47.237168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.237168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:47.237771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:47.238598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:47.239414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:47.239414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:47.239414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:52.142617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:52.143794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:52.143794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:52.144610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:52.178087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:52.178925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:52.179042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.180167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:52.180554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:52.180554 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.180554 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:52.180554 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.180554 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.180554 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:52.180554 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.180554 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:52.181276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.182900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.183660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:52.184518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:52.185447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.186034 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:52.186588 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.186588 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.187430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.187811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:52.188257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.188257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:52.190087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.190623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.191152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:52.191152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.191152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:52.191152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:52.191152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:52.191152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:52.191152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:52.191152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:52.191489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.191893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:49:52.191893 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.191893 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:52.191893 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.191893 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.191893 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:52.191893 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.191893 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:52.193297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:52.193842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.193842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:52.193842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.193842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:52.193842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.193842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.193842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:52.193842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.193842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.194228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:52.194780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.195182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.195182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:52.195182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.195182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:52.195623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.196117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:52.196714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.196714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:52.196714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.196714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.196714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:52.196714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.196714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:52.203190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:52.203709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:52.203846 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:52.204006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:52.204573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:52.204835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.205417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:49:52.205417 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.205417 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:52.205417 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.205417 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.205417 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:52.205417 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.205417 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:52.207374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.208003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:52.208003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.208003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:52.208003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.208003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.208003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:52.208003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.208003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:52.208749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.209428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.209833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:52.209833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.209833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.210611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:52.211113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.211685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.211685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:52.211685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.211685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:52.231434 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:52.232031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:52.232260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:52.232680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:52.233742 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:52.234072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:52.234239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.234958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:49:52.234958 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.234958 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:52.234958 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.234958 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.234958 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:52.234958 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.234958 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.236585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.237098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:52.237580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.237580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:52.238226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:52.239036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:52.239830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:52.239830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:52.239830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:57.143979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:57.144750 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:57.144750 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:57.145413 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:57.177477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:57.177804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.178525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:49:57.178525 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.178525 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:57.178525 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.178525 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.178525 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:57.178525 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.178525 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.179583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.180847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:57.182527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:57.183985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.185685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:57.185685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.185685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.185685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:57.185685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.185685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:57.185685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:57.185685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:57.185685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:57.185685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:57.186180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:57.186963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:57.187580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.188348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.188946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:57.190574 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:57.191020 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:57.191164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:57.191339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:49:57.192066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:57.192462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.193766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:49:57.193766 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.193766 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:57.193766 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.193766 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.193766 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:57.193766 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.193766 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:57.194673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:57.195026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.195026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:57.195026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.195026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:57.195026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.195026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.195026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:57.195026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.195026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.195541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:57.195845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.196151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.196151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:57.196151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.196151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:57.197452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.197835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:57.198350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.198350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:57.198350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.198350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.198350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:57.198350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.198350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:57.203758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:57.204520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:57.204672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:57.204841 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:57.205497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:57.206017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.206823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:49:57.206823 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.206823 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:57.206823 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.206823 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.206823 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:57.206823 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.206823 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:49:57.208116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.208576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:49:57.208576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.208576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:49:57.208576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.208576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.208576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:49:57.208576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.208576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:57.209546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.209925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.210351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:57.210351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.210351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.211398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:57.211957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.212486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.212486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:57.212486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.212486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:49:57.232061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:49:57.232540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:49:57.232635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:49:57.232839 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:49:57.233463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:57.233957 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:49:57.234302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.235143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:49:57.235143 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.235143 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:49:57.235143 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.235143 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.235143 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:49:57.235143 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.235143 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.236268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.236856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:49:57.237441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:49:57.238099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:49:57.238934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:49:57.238934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:49:57.238934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:49:57.238934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:49:57.238934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:49:57.238934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:02.144689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:02.145347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:02.145347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:02.146919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:02.182619 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:02.183375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:02.183584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:02.185124 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:02.185332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:02.185481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.186520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:02.186520 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.186520 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:02.186520 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.186520 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.186520 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:02.186520 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.186520 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.188762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.189489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.190144 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:02.191347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.192169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:02.193201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.193893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:02.194996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:02.195489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:02.196030 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:02.196433 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:02.196433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.196842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:02.196842 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.196842 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:02.196842 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.196842 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.196842 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:02.196842 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.196842 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:02.197684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:02.198283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.198283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:02.198283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.198283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:02.198283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.198283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.198283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:02.198283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.198283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.198784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:02.199172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.199580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.199580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:02.199580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.199580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:02.199880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.200408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:02.200981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.200981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:02.200981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.200981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.200981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:02.200981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.200981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:02.204579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:02.204876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:02.204967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:02.205026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:02.205481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:02.205739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.206404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:50:02.206404 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.206404 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:02.206404 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.206404 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.206404 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:02.206404 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.206404 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:02.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.208416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:02.209389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:02.210041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:02.210773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.211907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:02.232692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:02.233831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:02.233831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:02.233831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:02.234515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:02.234933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:02.235230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.236347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:50:02.236347 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.236347 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:02.236347 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.236347 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.236347 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:02.236347 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.236347 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.237860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.238283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:02.238619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.238619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:02.239389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:02.239802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:02.240338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:02.240338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:02.240338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:07.145251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:07.146295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:07.146295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:07.147169 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:07.181832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:07.182071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.182611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:07.182611 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.182611 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:07.182611 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.182611 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.182611 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:07.182611 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.182611 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.183752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.184598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:07.185436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:07.186113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:07.186684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:07.187065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:07.187065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.187065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:07.187065 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.187065 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:07.187065 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.187065 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.187065 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:07.187065 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.187065 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.187578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.187935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:07.188313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.188313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:07.188437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.188985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.189790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:07.189790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.189790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:07.194302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:07.194771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:07.194912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:07.195051 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:07.195890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:07.196230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.196880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:07.196880 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.196880 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:07.196880 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.196880 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.196880 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:07.196880 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.196880 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:07.198291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:07.198812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.198812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:07.198812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.198812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:07.198812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.198812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.198812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:07.198812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.198812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.199676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:07.200064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.200497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.200497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:07.200497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.200497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:07.201688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.202280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:07.202774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.202774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:07.202774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.202774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.202774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:07.202774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.202774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:07.205072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:07.205410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:07.205785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:07.205785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:07.206456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:07.207064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:07.210863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.212138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:07.212551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:07.212963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.213482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:07.214095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.214095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.214095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:07.214095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.214095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:07.233183 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:07.233741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:07.233933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:07.234236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:07.234919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:07.235450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:07.235768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.238728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:50:07.238728 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.238728 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:07.238728 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.238728 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.238728 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:07.238728 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.238728 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.240517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.241281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:07.241921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.241921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:07.242460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:07.243362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:07.244620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:07.244620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:07.244620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:12.145868 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:12.146664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:12.146664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:12.147364 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:12.180590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:12.181100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.181732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:12.181732 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.181732 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:12.181732 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.181732 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.181732 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:12.181732 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.181732 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.182753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.183355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:12.183979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:12.184706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.185658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:12.185658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.185658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.185658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:12.185658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.185658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:12.185658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:12.187316 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:12.187316 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:12.188323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:12.188323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:12.188323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.188717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:50:12.188717 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.188717 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:12.188717 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.188717 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.188717 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:12.188717 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.188717 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.189946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.190605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:12.191447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:12.192435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.193517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:12.193517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.193517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.193517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:12.193517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.193517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:12.194742 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:12.195159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:12.195412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:12.195604 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:12.196228 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:12.196551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.197415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:12.197415 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.197415 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:12.197415 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.197415 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.197415 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:12.197415 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.197415 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:12.199004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:12.199467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.199467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:12.199467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.199467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:12.199467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.199467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.199467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:12.199467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.199467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:12.201422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.201797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.201797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:12.201797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.201797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:12.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.203318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:12.203800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.203800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:12.203800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.203800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.203800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:12.203800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.203800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:12.205493 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:12.205745 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:12.205815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:12.205915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:12.206340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:12.206570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.207055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:50:12.207055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.207055 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:12.207055 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.207055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.207055 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:12.207055 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.207055 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:12.207779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.208385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:12.208385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.208385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:12.208385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.208385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.208385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:12.208385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.208385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:12.208710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.209081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.209544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:12.209544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.209544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.209785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:12.210278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.210828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.210828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:12.210828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.210828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:12.233826 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:12.234423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:12.234611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:12.234884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:12.235433 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:12.235802 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:12.236029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.236904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:50:12.236904 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.236904 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:12.236904 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.236904 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.236904 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:12.236904 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.236904 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.238602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.239269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:12.239987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.239987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:12.240248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:12.240931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:12.241783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:12.241783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:12.241783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:17.146524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:17.147476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:17.147606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:17.148832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:17.187491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:17.188322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:17.188567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:17.190076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:17.190342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:17.190577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.191319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:50:17.191319 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.191319 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:17.191319 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.191319 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.191319 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:17.191319 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.191319 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.193001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.193694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:17.194382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.194382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:17.195439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.196008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.196823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:17.196823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.196823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:17.196823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:17.196823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:17.196823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.197151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:17.197151 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.197151 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:17.197151 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.197151 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.197151 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:17.197151 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.197151 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.199114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.199484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:17.199827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.199827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:17.199827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:17.201731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.202191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.202732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:17.202732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.202732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:17.202732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:17.202732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:17.202732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:17.202928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.203474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:17.203474 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.203474 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:17.203474 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.203474 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.203474 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:17.203474 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.203474 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:17.204927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:17.205427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.205427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:17.205427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.205427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:17.205427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.205427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.205427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:17.205427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.205427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.206050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:17.206469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.206828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.206828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:17.206828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.206828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:17.207602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.208083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:17.208723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.208723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:17.208723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.208723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.208723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:17.208723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.208723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:17.208723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:17.208723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:17.208723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:17.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:17.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:17.209123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.209586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:17.209586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.209586 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:17.209586 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.209586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.209586 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:17.209586 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.209586 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:17.210483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:17.211066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.211066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:17.211066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.211066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:17.211066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.211066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.211066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:17.211066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.211066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:17.211450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.211450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:17.211450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:17.211450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:17.211450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.211450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:17.211450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:17.211450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:17.211450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:17.211450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.211450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:17.211801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:17.212488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:17.212898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.213433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:17.213433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:17.213433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.213433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:17.213433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.213433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:17.213433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.213433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.213433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:17.213433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.213433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:17.234237 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:17.234719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:17.234839 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:17.234928 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:17.235436 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:17.235840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:17.235840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.236294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:50:17.236294 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.236294 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:17.236294 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.236294 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.236294 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:17.236294 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.236294 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.237194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.237841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:17.238458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:17.239244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:17.240640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:17.240640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:17.240640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:17.240640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:17.240640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:17.240640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:22.147477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:22.148919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:22.148919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:22.149789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:22.184920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:22.185247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.185955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:50:22.185955 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.185955 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:22.185955 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.185955 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.185955 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:22.185955 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.185955 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.186946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.187687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:22.188416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:22.188862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.189427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:22.189427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.189427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.189427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:22.189427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.189427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:22.189427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:22.189427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:22.189427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:22.190064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:22.190408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:22.190660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.191484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:22.191484 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.191484 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:22.191484 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.191484 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.191484 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:22.191484 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.191484 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.193128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.193867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:22.194558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.194558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:22.194874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.195760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.196383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:22.196383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.196383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:22.198300 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:22.198641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:22.198789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:22.198902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:22.199305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:22.199485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.199963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:50:22.199963 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.199963 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:22.199963 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.199963 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.199963 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:22.199963 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.199963 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:22.201093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.201618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:22.201618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.201618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:22.201618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.201618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.201618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:22.201618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.201618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.202468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:22.203113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.203468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.203468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:22.203468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.203468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.203757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:22.204628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.205152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.205152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:22.205152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.205152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:22.208272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:22.209253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:22.209459 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:22.209756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:22.210410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:22.210900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.211870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:22.211870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.211870 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:22.211870 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.211870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.211870 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:22.211870 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.211870 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:22.213384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:22.213822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.213822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:22.213822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.213822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:22.213822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.213822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.213822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:22.213822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.213822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:22.214891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.214891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:22.214891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:22.214891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:22.214891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.214891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:22.214891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:22.214891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:22.214891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:22.214891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.214891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:22.215422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.215936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.215936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:22.215936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.215936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.216803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:22.217312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.217913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.217913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:22.217913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.217913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:22.234751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:22.235471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:22.235667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:22.235961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:22.236499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:22.237195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:22.237463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.238455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:50:22.238455 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.238455 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:22.238455 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.238455 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.238455 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:22.238455 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.238455 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.240041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.240689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:22.241396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.241396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:22.241988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:22.242855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:22.243849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:22.243849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:22.243849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:27.148098 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:27.148761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:27.148761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:27.149589 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:27.185895 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:27.186345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.187381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:50:27.187381 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.187381 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:27.187381 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.187381 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.187381 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:27.187381 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.187381 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.188613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.189390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:27.190749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:27.192200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.193491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:27.193491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.193491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.193491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:27.193491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.193491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:27.193491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:27.193491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:27.193491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:27.193491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:27.194605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:27.194605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.194605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:27.194605 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.194605 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:27.194605 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.194605 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.194605 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:27.194605 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.194605 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.195693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.196921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:27.197161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.197161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:27.197705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.198161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.198549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:27.198549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.198549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:27.198968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:27.198968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:27.199158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:27.199158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:27.199640 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:27.199948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.200742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:50:27.200742 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.200742 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:27.200742 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.200742 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.200742 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:27.200742 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.200742 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:27.201577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.201913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:27.201913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.201913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:27.201913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.201913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.201913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:27.201913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.201913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.202465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:27.202725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.203453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.203453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:27.203453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.203453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.203533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:27.203833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.204178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.204178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:27.204178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.204178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:27.208890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:27.209836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:27.209836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:27.209836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:27.210157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:27.210488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.214238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:27.214754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.215169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.215545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:27.216017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.216017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.216380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:27.217077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.217809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.217809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:27.217809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.217809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:27.235351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:27.235997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:27.236225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:27.236535 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:27.237028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:27.237308 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:27.237489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.238584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:50:27.238584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.238584 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:27.238584 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.238584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.238584 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:27.238584 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.238584 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.240170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.240778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:27.241689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.241689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:27.241816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:27.242815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:27.243980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:27.243980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:27.243980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:32.148846 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:32.149494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:32.149494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:32.150757 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:32.187273 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:32.187484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.188397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:50:32.188397 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.188397 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:32.188397 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.188397 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.188397 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:32.188397 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.188397 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.189103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.189698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:32.190270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:32.190923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.191740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:32.191740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.191740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.191740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:32.191740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.191740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:32.191740 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:32.191740 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:32.191740 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:32.192715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:32.192715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:32.192715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.194131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:32.194131 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.194131 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:32.194131 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.194131 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.194131 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:32.194131 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.194131 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.195495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.195851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:32.196182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.196182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:32.197787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.198360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.198869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:32.198869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.198869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:32.199179 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:32.199396 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:32.199481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:32.199570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:32.199920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:32.200307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.202655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:50:32.202655 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.202655 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:32.202655 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.202655 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.202655 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:32.202655 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.202655 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:32.205792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.206235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:32.206820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.207167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:32.207167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:32.207167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.207167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:32.207167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.207167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:32.207167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.207167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.207167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:32.207167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.207167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.207508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:32.207958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.208487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.208487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:32.208487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.208487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:32.209937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:32.210517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:32.210791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:32.211070 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:32.211715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:32.212268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.212934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:32.212934 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.212934 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:32.212934 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.212934 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.212934 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:32.212934 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.212934 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:32.214240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:32.214703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.214703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:32.214703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.214703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:32.214703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.214703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.214703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:32.214703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.214703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:32.215794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.215794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:32.215794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:32.215794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:32.215794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.215794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:32.215794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:32.215794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:32.215794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:32.215794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.215794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:32.216113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.216514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.216514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:32.216514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.216514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.217592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:32.218035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.218538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.218538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:32.218538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.218538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:32.236162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:32.236534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:32.236729 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:32.237005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:32.237756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:32.238150 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:32.238430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.239336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:50:32.239336 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.239336 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:32.239336 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.239336 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.239336 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:32.239336 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.239336 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.240682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.241471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:32.241993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.241993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:32.242631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:32.243363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:32.244097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:32.244097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:32.244097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:37.149397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:37.150254 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:37.150460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:37.151810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:37.186467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:37.186752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.187620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:50:37.187620 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.187620 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:37.187620 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.187620 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.187620 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:37.187620 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.187620 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.188704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.190404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:37.191811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:37.193368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.194421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:37.194421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.194421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.194421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:37.194421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.194421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:37.194421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:37.194421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:37.194421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:37.194421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:37.195035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:37.195035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.195035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:37.195035 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.195035 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:37.195035 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.195035 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.195035 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:37.195035 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.195035 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.195577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.196377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:37.196985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.196985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:37.197636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.198179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.198576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:37.198576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.198576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:37.199451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:37.199632 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:37.199699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:37.199782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:37.200255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:37.200537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.201199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:50:37.201199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.201199 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:37.201199 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.201199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.201199 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:37.201199 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.201199 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:37.201760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.202119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:37.202119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.202119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:37.202119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.202119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.202119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:37.202119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.202119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.202344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:37.202607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.202894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.202894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:37.202894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.202894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.203068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:37.203437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.203820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.203820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:37.203820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.203820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:37.210358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:37.210746 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:37.210975 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:37.211176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:37.211688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:37.212053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.212796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:37.212796 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.212796 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:37.212796 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.212796 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.212796 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:37.212796 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.212796 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:37.213749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:37.214171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.214171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:37.214171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.214171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:37.214171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.214171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.214171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:37.214171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.214171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:37.214628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.214628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:37.214628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:37.214628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:37.214628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.214628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:37.214628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:37.214628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:37.214628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:37.214628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.214628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:37.215020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.215441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.215441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:37.215441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.215441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.215905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:37.216878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.217780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.217780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:37.217780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.217780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:37.236513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:37.237135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:37.237372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:37.237689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:37.238534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:37.238989 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:37.239227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.240083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:50:37.240083 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.240083 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:37.240083 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.240083 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.240083 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:37.240083 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.240083 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.241657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.242268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:37.242898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.242898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:37.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:37.244441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:37.245280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:37.245280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:37.245280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:42.150487 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:42.151488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:42.151488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:42.152509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:42.155073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=434 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:50:42.184485 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:42.184777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.186382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:50:42.186382 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.186382 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:42.186382 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.186382 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.186382 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:42.186382 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.186382 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.187483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.188310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:42.189010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:42.189928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.191124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:42.191124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.191124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.191124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:42.191124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.191124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:42.191636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:42.191898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:42.192024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:42.192749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:42.193146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:42.193389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.193945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:42.193945 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.193945 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:42.193945 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.193945 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.193945 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:42.193945 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.193945 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.194782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.195138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:42.195492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.195492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:42.195747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.196189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.196708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:42.196708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.196708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:42.199796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:42.200050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:42.200142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:42.200260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:42.200673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:42.200894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.201649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:50:42.201649 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.201649 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:42.201649 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.201649 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.201649 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:42.201649 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.201649 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:42.202648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.203474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:42.203474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.203474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:42.203474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.203474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.203474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:42.203474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.203474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.203864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:42.204380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.204968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.204968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:42.204968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.204968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.205327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:42.205884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.206468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.206468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:42.206468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.206468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:42.210713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:42.210996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:42.211118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:42.211274 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:42.211668 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:42.211909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.212477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:42.212477 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.212477 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:42.212477 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.212477 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.212477 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:42.212477 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.212477 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:42.213396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:42.213918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.213918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:42.213918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.213918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:42.213918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.213918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.213918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:42.213918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.213918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:42.214288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.214288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:42.214288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:42.214288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:42.214288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.214288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:42.214288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:42.214288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:42.214288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:42.214288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.214288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:42.214705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.215251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.215251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:42.215251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.215251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.215405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:42.216025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.216687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.216687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:42.216687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.216687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:42.237033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:42.237580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:42.237675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:42.237877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:42.238443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:42.238885 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:42.239156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.239965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:50:42.239965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.239965 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:42.239965 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.239965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.239965 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:42.239965 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.239965 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.240924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.241423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:42.241870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.241870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:42.241957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:42.242559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:42.243399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:42.243399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:42.243399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:47.151066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:47.152560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:47.152560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:47.152560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:47.188934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:47.189225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.190559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:50:47.190559 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.190559 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:47.190559 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.190559 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.190559 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:47.190559 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.190559 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.191229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.192021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:47.193486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:47.194581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.195695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:47.195695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.195695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.195695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:47.195695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.195695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:47.195695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:47.195695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:47.195695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:47.195695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:47.196425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:47.197406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:47.198454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.199079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.199788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:47.200142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:47.200447 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:47.200820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:47.200820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:47.201116 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:47.201518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.202108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:50:47.202108 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.202108 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:47.202108 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.202108 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.202108 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:47.202108 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.202108 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:47.203417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.203837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:47.203837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.203837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:47.203837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.203837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.203837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:47.203837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.203837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.204877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:47.205301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.205591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.205591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:47.205591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.205591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.206916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:47.207701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.208354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.208354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:47.208354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.208354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:47.211255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:47.211496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:47.211617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:47.211752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:47.212079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:47.212431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.212763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:47.212763 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.212763 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:47.212763 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.212763 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.212763 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:47.212763 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.212763 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:47.213536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:47.213983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.213983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:47.213983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.213983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:47.213983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.213983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.213983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:47.213983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.213983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:47.214531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.214531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:47.214531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:47.214531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:47.214531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.214531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:47.214531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:47.214531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:47.214531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:47.214531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.214531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:47.214934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.215383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.215383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:47.215383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.215383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.215648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:47.216325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.216903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.216903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:47.216903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.216903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:47.237637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:47.238469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:47.238469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:47.238804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:47.239726 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:47.240143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:47.240459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.241462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:50:47.241462 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.241462 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:47.241462 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.241462 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.241462 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:47.241462 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.241462 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.243113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.243857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:47.244499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.244499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:47.244918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:47.245711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:47.246666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:47.246666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:47.246666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:52.151723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:52.152819 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:52.152819 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:52.153704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:52.188594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:52.189040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.189172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:50:52.189172 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.189172 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:52.189172 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.189172 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.189172 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:52.189172 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.189172 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.189988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.190713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:52.191251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:52.191894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.192932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:52.192932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.192932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.192932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:52.192932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.192932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:52.192932 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:52.193310 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:52.193649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:52.194339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:52.194524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:52.194524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.195040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:52.195040 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.195040 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:52.195040 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.195040 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.195040 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:52.195040 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.195040 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.196868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.198310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:52.198693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.198693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:52.199292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.199705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.200165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:52.200165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.200165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:52.200411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:52.200979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:52.201031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:52.201206 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:52.201713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:52.202092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.202878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:50:52.202878 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.202878 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:52.202878 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.202878 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.202878 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:52.202878 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.202878 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:52.204000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.204459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:52.204459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.204459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:52.204459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.204459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.204459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:52.204459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.204459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.205238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:52.205559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.206242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.206242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:52.206242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.206242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.206834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:52.208020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.209193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.209193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:52.209193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.209193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:52.211567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:52.212555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:52.212713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:52.212884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:52.213305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:52.213543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.214089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:52.214089 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.214089 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:52.214089 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.214089 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.214089 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:52.214089 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.214089 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:52.215029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:52.215465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.215465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:52.215465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.215465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:52.215465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.215465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.215465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:52.215465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.215465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:52.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:52.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:52.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:52.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:52.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:52.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:52.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:52.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:52.216868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.217627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.217627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:52.217627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.217627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.219522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:52.220015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.220477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.220477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:52.220477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.220477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:52.238257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:52.238715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:52.238842 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:52.239126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:52.239665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:52.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:52.240382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.241150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:50:52.241150 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.241150 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:52.241150 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.241150 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.241150 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:52.241150 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.241150 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.242621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.243279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:52.244316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:52.245438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:52.246774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:52.246774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:52.246774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:52.246774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:52.246774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:52.246774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:57.152288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:57.153375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:57.153375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:57.153770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:57.193527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:57.194104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.194654 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:57.195739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:57.196236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:50:57.196236 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.196236 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:57.196236 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.196236 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.196236 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:57.196236 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.196236 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:57.196811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.198434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.199261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:57.200144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.200144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:57.200144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:57.200453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.202413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.204478 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.206902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:57.207851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:57.208436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:50:57.208898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.209242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.209735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.211044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:57.212200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.213242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:57.214067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:57.214659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.215230 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:50:57.215672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:50:57.216082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.216082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:50:57.216082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.216082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:50:57.216082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.216082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.216082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:50:57.216082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.216082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:50:57.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:57.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:57.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:57.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:57.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:57.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:57.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:57.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:57.217382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.217736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.217736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:57.217736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.217736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.219500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:57.219948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.220453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.220453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:57.220453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.220453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:50:57.238763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:50:57.239315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:50:57.239488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:50:57.239570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:50:57.240951 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:57.241478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.245699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.246697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:50:57.247367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.247367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:50:57.247367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.247367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.247367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:50:57.247367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.247367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:50:57.247851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:50:57.248260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:50:57.248671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:50:57.248671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:50:57.248671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:02.153044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:02.154533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:02.154533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:02.155054 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:02.188142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:02.188477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.189121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:02.189121 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.189121 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:02.189121 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.189121 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.189121 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:02.189121 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.189121 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.189977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.190842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:02.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:02.192790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.193905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:02.193905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.193905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.193905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:02.193905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.193905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:02.195757 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:02.196312 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:02.196446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:02.197166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:02.197410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:02.197844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.198653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:51:02.198653 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.198653 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:02.198653 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.198653 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.198653 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:02.198653 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.198653 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.199734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.200179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:02.200646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.200646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:02.200965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.201491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.202279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:02.202279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.202279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:02.202669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:02.203164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:02.203403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:02.203611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:02.204164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:02.204369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.205000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:02.205000 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.205000 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:02.205000 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.205000 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.205000 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:02.205000 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.205000 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:02.206646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.207436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:02.207436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.207436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:02.207436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.207436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.207436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:02.207436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.207436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.208063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:02.208555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.208948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.208948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:02.208948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.208948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.209973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:02.210731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.211277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.211277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:02.211277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.211277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:02.212465 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:02.212913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:02.213111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:02.213391 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:02.213934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:02.214173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.214976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:51:02.214976 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.214976 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:02.214976 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.214976 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.214976 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:02.214976 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.214976 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:02.216308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:02.216780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.216780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:02.216780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.216780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:02.216780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.216780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.216780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:02.216780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.216780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:02.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:02.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:02.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:02.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:02.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:02.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:02.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:02.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.217912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:02.218352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.219154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.219154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:02.219154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.219154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.220645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:02.221160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.221786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.221786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:02.221786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.221786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:02.239648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:02.240120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:02.240412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:02.240744 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:02.241494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:02.241893 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:02.242178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.243000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:51:02.243000 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.243000 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:02.243000 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.243000 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.243000 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:02.243000 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.243000 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.244751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.245492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:02.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:02.246829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:02.247327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:02.247855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:02.247855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:02.247855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:07.153642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:07.154495 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:07.154495 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:07.155690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:07.196204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:07.196716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.197407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:07.198257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:07.198257 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.198257 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:07.198257 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.198257 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.198257 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:07.198257 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.198257 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.199351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.199789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:07.200151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:07.200625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.201108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:07.201108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.201108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.201108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:07.201108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.201108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:07.201108 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:07.201108 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:07.201594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:07.201987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:07.202276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.203092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:07.203433 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:07.203536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:07.203660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:07.204016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:51:07.204016 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.204016 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:07.204016 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.204016 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.204016 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:07.204016 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.204016 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.206286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.206663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:07.207008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.207008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:07.207008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:07.208868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.209353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.209889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:07.209889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.209889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:07.209889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.210481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:07.210481 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.210481 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:07.210481 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.210481 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.210481 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:07.210481 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.210481 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:07.211573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.212046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:07.212046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.212046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:07.212046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.212046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.212046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:07.212046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.212046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.213358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:07.213803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.214197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.214197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:07.214197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.214197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:07.214197 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.216037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:07.216602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.217164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.217164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:07.217164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.217164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:07.217164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:07.217164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:07.217164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:07.218177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:07.218440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.219089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:51:07.219089 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.219089 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:07.219089 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.219089 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.219089 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:07.219089 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.219089 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:07.220611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:07.221119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.221119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:07.221119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.221119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:07.221119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.221119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.221119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:07.221119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.221119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:07.222109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.222109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:07.222109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:07.222109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:07.222109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.222109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:07.222109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:07.222109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:07.222109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:07.222109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.222109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:07.222517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.223013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.223013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:07.223013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.223013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.224008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:07.224602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.225472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.225472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:07.225472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.225472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:07.240199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:07.241028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:07.241267 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:07.241557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:07.242397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:07.242785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:07.243034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:51:07.244156 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.244156 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:07.244156 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.244156 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.244156 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:07.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.246167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.246575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:07.246902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.246902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:07.248121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:07.248651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:07.249501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:07.249501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:07.249501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:12.154236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:12.155100 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:12.155100 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:12.156255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:12.191305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:12.191638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.192480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:12.192480 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.192480 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:12.192480 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.192480 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.192480 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:12.192480 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.192480 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.193537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.194126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:12.194565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:12.195034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.195842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:12.195842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.195842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.195842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:12.195842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.195842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:12.199042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:12.199400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:12.199497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:12.200198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:12.200435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:12.200606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.201430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:51:12.201430 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.201430 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:12.201430 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.201430 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.201430 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:12.201430 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.201430 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.202349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.202796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:12.203354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:12.203927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.204591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:12.204591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.204591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.204591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:12.204591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.204591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:12.204591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:12.204591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:12.204591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:12.204591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:12.204911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:12.204911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.205257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:12.205257 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.205257 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:12.205257 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.205257 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.205257 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:12.205257 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.205257 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:12.206022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.206864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:12.206864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.206864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:12.206864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.206864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.206864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:12.206864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.206864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.207279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:12.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.208140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.208140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:12.208140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.208140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.208506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:12.209042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.209705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.209705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:12.209705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.209705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:12.214501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:12.215113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:12.215344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:12.215560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:12.215992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:12.216489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.217303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:51:12.217303 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.217303 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:12.217303 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.217303 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.217303 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:12.217303 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.217303 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:12.219133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:12.219578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.219578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:12.219578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.219578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:12.219578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.219578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.219578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:12.219578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.219578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:12.220837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.220837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:12.220837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:12.220837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:12.220837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.220837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:12.220837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:12.220837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:12.220837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:12.220837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.220837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:12.221182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.221614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.221614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:12.221614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.221614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.222728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:12.223838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.224789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.224789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:12.224789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.224789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:12.240956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:12.241506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:12.241745 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:12.242391 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:12.242847 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:12.243239 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:12.243498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.244499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:51:12.244499 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.244499 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:12.244499 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.244499 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.244499 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:12.244499 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.244499 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.245927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.246557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:12.247167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.247167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:12.248516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:12.249193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:12.249733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:12.249733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:12.249733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:17.155014 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:17.155962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:17.155962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:17.156929 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:17.194684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:17.195191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.196346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:17.196346 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.196346 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:17.196346 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.196346 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.196346 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:17.196346 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.196346 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.197260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.197678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:17.198085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:17.198629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.199584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:17.199584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.199584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.199584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:17.199584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.199584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:17.199584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:17.200991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:17.200991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:17.201427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:17.201658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:17.201826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:17.203375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:17.204025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.204486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:17.204486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.204486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.204486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:17.204486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.204486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:17.204660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.205111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.205661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:17.205661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.205661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:17.205661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:17.205661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:17.205661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:17.205661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:17.205990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:17.206238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.206649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:17.206649 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.206649 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:17.206649 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.206649 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.206649 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:17.206649 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.206649 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:17.207392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.207939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:17.207939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.207939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:17.207939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.207939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.207939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:17.207939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.207939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.208204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:17.208619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.208970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.208970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:17.208970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.208970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.209283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:17.209802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.210614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.210614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:17.210614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.210614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:17.214960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:17.215510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:17.215510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:17.215628 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:17.216160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:17.216780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.217789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:51:17.217789 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.217789 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:17.217789 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.217789 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.217789 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:17.217789 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.217789 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:17.219174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:17.219651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.219651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:17.219651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.219651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:17.219651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.219651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.219651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:17.219651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.219651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.220979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:17.221382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.221763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.221763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:17.221763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.221763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.222147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:17.222682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:17.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:17.241481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:17.242080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:17.242315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:17.242604 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:17.243201 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:17.243489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:17.243660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.244556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:51:17.244556 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.244556 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:17.244556 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.244556 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.244556 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:17.244556 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.244556 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.246173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:17.246956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.246956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:17.248391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:17.248871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:17.249409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:17.249409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:17.249409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:22.155708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:22.156277 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:22.156349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:22.157170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:22.194557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:22.195062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.196401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:22.196401 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.196401 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:22.196401 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.196401 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.196401 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:22.196401 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.196401 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.197192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.197645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:22.198063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.198063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:22.198963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.199645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.200293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:22.200293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.200293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:22.200410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:22.201061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:22.201061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:22.201947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:22.202157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:22.202323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.203079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:51:22.203079 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.203079 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:22.203079 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.203079 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.203079 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:22.203079 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.203079 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.204112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.204487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:22.204823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.204823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:22.205283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.205742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.206285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:22.206285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.206285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:22.206285 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:22.207951 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:22.208058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:22.208192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:22.209380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:22.209621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.210975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:22.210975 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.210975 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:22.210975 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.210975 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.210975 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:22.210975 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.210975 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:22.212201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.212742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:22.212742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.212742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:22.212742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.212742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.212742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:22.212742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.212742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.213932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:22.214351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.214684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.214684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:22.214684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.214684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.215709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:22.216427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.217230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.217230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:22.217230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.217230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:22.217230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:22.217406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:22.217458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:22.217598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:22.218133 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:22.218414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.219250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:22.219250 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.219250 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:22.219250 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.219250 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.219250 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:22.219250 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.219250 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:22.220286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.221363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:22.221782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:22.222041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:22.222390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.222839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:22.242352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:22.242924 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:22.243120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:22.243413 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:22.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:22.244329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:22.244628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.245405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:51:22.245405 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.245405 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:22.245405 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.245405 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.245405 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:22.245405 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.245405 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.246856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.247435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:22.248031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:22.248542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:22.249044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:22.249044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:22.249044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:22.249044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:22.249044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:22.249044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:27.157268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:27.157268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:27.157268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:27.158441 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:27.194658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:27.194839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.195513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:27.195513 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.195513 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:27.195513 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.195513 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.195513 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:27.195513 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.195513 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.196715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.197570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:27.198608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:27.199538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.201500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:27.201500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.201500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.201500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:27.201500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.201500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:27.201500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:27.202819 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:27.203136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:27.204132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:27.204538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:27.204882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.205798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:51:27.205798 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.205798 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:27.205798 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.205798 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.205798 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:27.205798 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.205798 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.207664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.208341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:27.208933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.208933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:27.208933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:27.211108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.211919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.213084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:27.213084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.213084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:27.213084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:27.213084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:27.213084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:27.213084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:27.213084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.213765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:27.213765 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.213765 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:27.213765 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.213765 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.213765 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:27.213765 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.213765 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:27.215063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:27.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:27.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:27.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.215939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:27.216340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.216718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.216718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:27.216718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.216718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.217044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:27.217602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.218162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.218162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:27.218162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.218162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:27.218162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:27.218652 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:27.218838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:27.219040 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:27.219723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:27.220116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.220901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:51:27.220901 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.220901 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:27.220901 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.220901 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.220901 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:27.220901 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.220901 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:27.222023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:27.222812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.222812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:27.222812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.222812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:27.222812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.222812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.222812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:27.222812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.222812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:27.223539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.223539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:27.223539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:27.223539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:27.223539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.223539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:27.223539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:27.223539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:27.223539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:27.223539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.223539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:27.224070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.224698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.224698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:27.224698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.224698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.225065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:27.225645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.226178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.226178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:27.226178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.226178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:27.242860 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:27.243440 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:27.243609 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:27.243913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:27.244585 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:27.244992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:27.245236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.246146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:51:27.246146 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.246146 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:27.246146 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.246146 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.246146 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:27.246146 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.246146 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.247532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.248142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:27.248815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.248815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:27.249120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:27.250126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:27.250807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:27.250807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:27.250807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:32.157113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:32.158082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:32.158082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:32.159584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:32.199015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:32.199323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.200073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:32.200073 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.200073 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:32.200073 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.200073 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.200073 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:32.200073 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.200073 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.201331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.201873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:32.202336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.202336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:32.202534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.203116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.203733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:32.203733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.203733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:32.203733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:32.203733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:32.203733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:32.204477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:32.204976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:32.205345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.206228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:51:32.206228 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.206228 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:32.206228 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.206228 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.206228 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:32.206228 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.206228 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.207373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.207709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:32.208033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.208033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:32.208411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.208814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.209308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:32.209308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.209308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:32.210145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:32.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:32.210499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:32.210631 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:32.211045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:32.211288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.212067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:32.212067 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.212067 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:32.212067 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.212067 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.212067 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:32.212067 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.212067 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:32.213386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.213816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:32.213816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.213816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:32.213816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.213816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.213816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:32.213816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.213816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.214893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:32.215253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.215742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.215742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:32.215742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.215742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.216841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:32.217277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.217654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.217654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:32.217654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.217654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:32.218747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:32.219166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:32.219323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:32.219470 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:32.219994 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:32.220402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.221102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:51:32.221102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.221102 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:32.221102 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.221102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.221102 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:32.221102 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.221102 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:32.221859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:32.222283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.222283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:32.222283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.222283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:32.222283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.222283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.222283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:32.222283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.222283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:32.222764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.222764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:32.222764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:32.222764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:32.222764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.222764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:32.222764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:32.222764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:32.222764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:32.222764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.222764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:32.223100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.223497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.223497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:32.223497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.223497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.223849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:32.224302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.224639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.224639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:32.224639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.224639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:32.243395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:32.243870 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:32.243962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:32.244122 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:32.244669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:32.245019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:32.245315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.246236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:51:32.246236 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.246236 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:32.246236 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.246236 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.246236 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:32.246236 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.246236 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.247229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.247676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:32.248128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.248128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:32.248294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:32.248774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:32.249242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:32.249242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:32.249242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:37.157760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:37.158549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:37.158549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:37.159169 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:37.191632 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:37.191757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.192716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:37.192716 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.192716 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:37.192716 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.192716 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.192716 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:37.192716 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.192716 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.193595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.194358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:37.195043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:37.195993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.197032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:37.197032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.197032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.197032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:37.197032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.197032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:37.203302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:37.203832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:37.203960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:37.205023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:37.205456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:37.205797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.206687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:51:37.206687 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.206687 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:37.206687 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.206687 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.206687 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:37.206687 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.206687 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.208281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.209005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:37.209613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.209613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:37.209972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.211119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.212008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:37.212008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.212008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:37.212008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:37.212008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:37.212008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:37.212008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:37.212428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:37.212497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.213290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:37.213290 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.213290 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:37.213290 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.213290 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.213290 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:37.213290 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.213290 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:37.214560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.215066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:37.215066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.215066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:37.215066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.215066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.215066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:37.215066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.215066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.216067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:37.216464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.216853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.216853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:37.216853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.216853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.217416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:37.217911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.218474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.218474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:37.218474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.218474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:37.219242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:37.219786 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:37.220081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:37.220404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:37.220891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:37.221266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.222027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:51:37.222027 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.222027 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:37.222027 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.222027 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.222027 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:37.222027 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.222027 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:37.223387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:37.223969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.223969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:37.223969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.223969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:37.223969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.223969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.223969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:37.223969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.223969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:37.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:37.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:37.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:37.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:37.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:37.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:37.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:37.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.225085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:37.225511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.225948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.225948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:37.225948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.225948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.227620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:37.228194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.228743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.228743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:37.228743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.228743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:37.243957 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:37.245012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:37.245012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:37.245012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:37.245401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:37.245778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:37.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.247340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:51:37.247340 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.247340 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:37.247340 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.247340 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.247340 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:37.247340 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.247340 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.248071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.248494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:37.248868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.248868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:37.249279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:37.249732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:37.250256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:37.250256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:37.250256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:42.158602 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:42.159665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:42.159665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:42.160662 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:42.163371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:51:42.198881 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:42.199885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.201351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:42.201351 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.201351 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:42.201351 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.201351 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.201351 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:42.201351 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.201351 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.202250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.203316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:42.204361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:42.204902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.205498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:42.205498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.205498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.205498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:42.205498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.205498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:42.205498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:42.205498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:42.205498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:42.205802 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:42.206024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:42.206024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.206811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:51:42.206811 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.206811 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:42.206811 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.206811 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.206811 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:42.206811 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.206811 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.208425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.209450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:42.210255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.210255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:42.210388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.211085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.211557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:42.211557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.211557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:42.211557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:42.211690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:42.212233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:42.212233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:42.212544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:42.212801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.213361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:42.213361 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.213361 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:42.213361 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.213361 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.213361 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:42.213361 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.213361 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:42.214179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.214881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:42.214881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.214881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:42.214881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.214881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.214881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:42.214881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.214881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.215331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:42.215561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.215819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.215819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:42.215819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.215819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.216085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:42.217170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.217939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.217939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:42.217939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.217939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:42.219701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:42.220219 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:42.220524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:42.220805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:42.221217 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:42.221476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.222249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:51:42.222249 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.222249 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:42.222249 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.222249 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.222249 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:42.222249 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.222249 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:42.223760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:42.224275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.224275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:42.224275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.224275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:42.224275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.224275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.224275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:42.224275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.224275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:42.225564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.225564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:42.225564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:42.225564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:42.225564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.225564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:42.225564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:42.225564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:42.225564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:42.225564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.225564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:42.225974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.226448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.226448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:42.226448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.226448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.227009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:42.227699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.228441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.228441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:42.228441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.228441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:42.244501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:42.245181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:42.245523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:42.245973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:42.246698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:42.247004 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:42.247200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.248036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:51:42.248036 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.248036 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:42.248036 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.248036 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.248036 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:42.248036 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.248036 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.249817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.250677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:42.251560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.251560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:42.252172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:42.252975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:42.253536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:42.253536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:42.253536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:47.159164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:47.159813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:47.159885 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:47.160905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:47.204652 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:47.205671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:47.205812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:47.207464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:47.207920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.209047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:47.209047 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.209047 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:47.209047 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.209047 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.209047 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:47.209047 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.209047 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.210583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.211106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:47.211614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.211614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:47.211614 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:47.212057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.212734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.213474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:47.213474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.213474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:47.213474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:47.213474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:47.213474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.213751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:47.214437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:51:47.214437 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.214437 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:47.214437 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.214437 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.214437 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:47.214437 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.214437 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:47.214783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.216657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.217158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:47.217656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.217656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:47.217656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:47.219026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.219490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.219894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:47.219894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.219894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:47.219894 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:47.220439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.221097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:47.221097 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.221097 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:47.221097 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.221097 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.221097 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:47.221097 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.221097 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:47.222237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.222575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:47.222575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.222575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:47.222575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.222575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.222575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:47.222575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.222575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:47.222575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.224017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:47.224343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.224613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.224613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:47.224613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.224613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:47.224613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.225699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:47.226092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.226537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.226537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:47.226537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.226537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:47.226537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:47.226537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:47.227227 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:47.227468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:51:47.228106 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.228106 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:47.228106 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.228106 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.228106 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:47.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:47.229702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:47.230160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.230160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:47.230160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.230160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:47.230160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.230160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.230160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:47.230160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.230160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:47.231225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.231225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:47.231225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:47.231225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:47.231225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.231225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:47.231225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:47.231225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:47.231225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:47.231225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.231225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:47.231579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.231959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.231959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:47.231959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.231959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.232424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:47.233035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.233786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.233786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:47.233786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.233786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:47.244904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:47.245527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:47.245743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:47.246090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:47.246685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:47.247083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:47.247374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.248235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:51:47.248235 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.248235 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:47.248235 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.248235 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.248235 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:47.248235 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.248235 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.250043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.250905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:47.251612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:47.252440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:47.253403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:47.253403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:47.253403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:47.253403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:47.253403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:47.253403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:52.160017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:52.160958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:52.160958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:52.162147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:52.201841 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:52.202368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.203514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:52.203514 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.203514 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:52.203514 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.203514 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.203514 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:52.203514 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.203514 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.204483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.204898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:52.205252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.205252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:52.205736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.206108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.206677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:52.206677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.206677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:52.206677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:52.206677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:52.206677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:52.207917 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:52.208277 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:52.208511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.209101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:52.209101 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.209101 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:52.209101 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.209101 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.209101 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:52.209101 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.209101 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.210259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.210890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:52.211529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.211529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:52.211878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.212523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.213042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:52.213042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.213042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:52.213422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:52.213880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:52.213981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:52.214092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:52.214546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:52.214735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.215306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:52.215306 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.215306 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:52.215306 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.215306 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.215306 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:52.215306 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.215306 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:52.216235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.216744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:52.216744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.216744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:52.216744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.216744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.216744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:52.216744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.216744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:52.217825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.218514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:52.219094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.219780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:52.219780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:52.219780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.219780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:52.219780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.219780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:52.219780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.219780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.219780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:52.219780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.219780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:52.223198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:52.223809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:52.223987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:52.224192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:52.224718 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:52.224950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:51:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:52.227095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:52.227752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.227752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:52.227752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.227752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:52.227752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.227752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.227752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:52.227752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.227752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:52.228503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.228503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:52.228503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:52.228503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:52.228503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.228503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:52.228503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:52.228503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:52.228503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:52.228503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.228503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:52.228996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.229578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.229578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:52.229578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.229578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.229735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:52.230391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.231111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.231111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:52.231111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.231111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:52.245481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:52.246239 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:52.246451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:52.246733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:52.247412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:52.247827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:52.248059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.248990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:51:52.248990 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.248990 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:52.248990 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.248990 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.248990 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:52.248990 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.248990 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.250597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.251256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:52.251852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.251852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:52.252398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:52.253149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:52.254012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:52.254012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:52.254012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:57.161446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:57.162332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:57.162332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:57.163711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:57.200006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:57.200451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.201282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:51:57.201282 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.201282 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:57.201282 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.201282 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.201282 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:57.201282 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.201282 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.201941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.202388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:57.202800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:57.203303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.203878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:57.203878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.203878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.203878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:57.203878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.203878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:57.206636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:57.206794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:57.206852 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:57.207506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:57.207793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:57.208068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.208937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:51:57.208937 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.208937 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:57.208937 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.208937 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.208937 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:57.208937 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.208937 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.209824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.210193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:57.210652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.210652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.210774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.211289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:57.211885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.211885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:57.213896 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:57.214790 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:57.214790 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:57.214790 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:51:57.215107 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:57.215425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.216126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:51:57.216126 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.216126 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:57.216126 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.216126 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.216126 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:57.216126 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.216126 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:57.217070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:57.217628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.217628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:57.217628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.217628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:57.217628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.217628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.217628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:57.217628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.217628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.218384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:57.218944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.219347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.219347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:57.219347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.219347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:57.220225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.220690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:57.221257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.221257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:57.221257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.221257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.221257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:57.221257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.221257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:57.223738 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:57.224246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:57.224384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:57.224551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:57.225126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:57.225507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.226221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:51:57.226221 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.226221 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:57.226221 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.226221 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.226221 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:57.226221 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.226221 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:51:57.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:51:57.227645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.227645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:51:57.227645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.227645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:51:57.227645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.227645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.227645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:51:57.227645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.227645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:51:57.228282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.228282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:57.228282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:57.228282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:57.228282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.228282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:57.228282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:57.228282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:57.228282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:57.228282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.228282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:57.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.229035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.229035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:57.229035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.229035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.229759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:57.230406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.230866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.230866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:57.230866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.230866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:51:57.246105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:51:57.246989 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:51:57.246989 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:51:57.247237 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:51:57.247855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:57.248323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:51:57.248566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.249574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:51:57.249574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.249574 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:51:57.249574 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.249574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.249574 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:51:57.249574 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.249574 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.251549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:51:57.252084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.252084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:51:57.252569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:51:57.253353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:51:57.254298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:51:57.254298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:51:57.254298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:02.161672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:02.162583 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:02.162583 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:02.163545 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:02.203658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:02.203995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.204800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:02.204800 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.204800 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:02.204800 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.204800 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.204800 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:02.204800 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.204800 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.205503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.206365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:02.207012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:02.207778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.208785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:02.208785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.208785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.208785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:02.208785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.208785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:02.208785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:02.208785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:02.208785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:02.208785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:02.209341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:02.209341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.209630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:02.209630 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.209630 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:02.209630 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.209630 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.209630 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:02.209630 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.209630 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.210395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.210736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.211153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:02.211681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.212342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.212342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:02.212342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.212342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:02.214488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:02.215222 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:02.215427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:02.215643 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:02.216057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:02.216237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.216821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:52:02.216821 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.216821 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:02.216821 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.216821 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.216821 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:02.216821 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.216821 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:02.218327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:02.218779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.218779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:02.218779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.218779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:02.218779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.218779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.218779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:02.218779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.218779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.219327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:02.219679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.220121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.220121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:02.220121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.220121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:02.220613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.221068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:02.221608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.221608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:02.221608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.221608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.221608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:02.221608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.221608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:02.224157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:02.224692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:02.224877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:02.225101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:02.225644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:02.226009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.226977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:52:02.226977 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.226977 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:02.226977 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.226977 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.226977 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:02.226977 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.226977 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:02.228578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:02.229141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.229141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:02.229141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.229141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:02.229141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.229141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.229141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:02.229141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.229141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:02.230154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.230154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:02.230154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:02.230154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:02.230154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.230154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:02.230154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:02.230154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:02.230154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:02.230154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.230154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:02.230593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.231010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.231010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:02.231010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.231010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.232202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:02.232802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.233363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.233363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:02.233363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.233363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:02.246936 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:02.247402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:02.247784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:02.248013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:02.248785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:02.249146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:02.249402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.250394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:52:02.250394 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.250394 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:02.250394 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.250394 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.250394 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:02.250394 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.250394 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.251744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.252615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:02.253178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.253178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:02.253477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:02.254480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:02.255073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:02.255073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:02.255073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:07.162369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:07.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:07.163291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:07.164784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:07.196818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:07.197129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.198046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:07.198046 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.198046 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:07.198046 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.198046 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.198046 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:07.198046 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.198046 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.198926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.199540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:07.200263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:07.201001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.201923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:07.201923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.201923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.201923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:07.201923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.201923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:07.207783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:07.208276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:07.208422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:07.209302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:07.209626 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:07.209881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.210683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:07.210683 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.210683 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:07.210683 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.210683 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.210683 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:07.210683 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.210683 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.212239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.212591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:07.212921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.212921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.213431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.214511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:07.215409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.215409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:07.215409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:07.215821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:07.215821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:07.215821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:07.216518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:07.216789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.217577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:52:07.217577 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.217577 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:07.217577 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.217577 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.217577 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:07.217577 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.217577 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:07.218452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:07.218973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.218973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:07.218973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.218973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:07.218973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.218973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.218973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:07.218973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.218973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.219289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:07.219744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.220148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.220148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:07.220148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.220148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:07.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.220818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:07.221384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.221384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:07.221384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.221384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.221384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:07.221384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.221384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:07.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:07.225071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:07.225203 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:07.225354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:07.225862 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:07.226231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.227001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:52:07.227001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.227001 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:07.227001 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.227001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.227001 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:07.227001 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.227001 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:07.228200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:07.228692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.228692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:07.228692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.228692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:07.228692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.228692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.228692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:07.228692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.228692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:07.230157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.230157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:07.230157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:07.230157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:07.230157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.230157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:07.230157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:07.230157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:07.230157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:07.230157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.230157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:07.230718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.231128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.231128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:07.231128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.231128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.232135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:07.232681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.233274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.233274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:07.233274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.233274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:07.247450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:07.247903 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:07.248045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:07.248401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:07.248926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:07.249333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:07.249569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.250302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:07.250302 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.250302 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:07.250302 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.250302 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.250302 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:07.250302 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.250302 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.251850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:07.252510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:07.253076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:07.253831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:07.254669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:07.254669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:07.254669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:12.163033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:12.163820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:12.163820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:12.164992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:12.201580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:12.201990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.202769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:12.202769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.202769 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:12.202769 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.202769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.202769 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:12.202769 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.202769 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.203597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.205409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:12.205930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:12.206758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.207557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:12.207557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.207557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.207557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:12.207557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.207557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:12.208419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:12.209397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:12.209397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:12.209784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:12.210013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:12.210171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:12.213585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.214052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:12.214378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:12.214691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:12.215055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.215055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:12.215055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.215055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:12.215055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.215055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.215055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:12.215055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.215055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:12.215414 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:12.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:12.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:12.216074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:12.216509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:12.216699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:52:12.217404 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.217404 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:12.217404 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.217404 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.217404 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:12.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.217404 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:12.218408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:12.218840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.218840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:12.218840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.218840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:12.218840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.218840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.218840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:12.218840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.218840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.219332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:12.219637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.219918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.219918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:12.219918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.219918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:12.220381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:12.221084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.221084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:12.221084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.221084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.221084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:12.221084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.221084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:12.225205 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:12.225656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:12.225790 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:12.225948 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:12.226454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:12.226848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.227658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:52:12.227658 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.227658 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:12.227658 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.227658 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.227658 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:12.227658 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.227658 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:12.229155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:12.229605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.229605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:12.229605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.229605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:12.229605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.229605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.229605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:12.229605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.229605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:12.230966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.230966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:12.230966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:12.230966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:12.230966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.230966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:12.230966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:12.230966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:12.230966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:12.230966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.230966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:12.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.231777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.231777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:12.231777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.231777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.232670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:12.233818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.235111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.235111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:12.235111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.235111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:12.247794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:12.248466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:12.248532 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:12.248707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:12.249434 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:12.249859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:12.250098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.251076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:12.251076 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.251076 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:12.251076 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.251076 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.251076 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:12.251076 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.251076 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.252357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:12.252798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.253261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:12.254130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:12.254698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:12.255264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:12.255264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:17.163625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:17.164332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:17.164633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:17.166128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:17.201823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:17.202059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.202695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:17.202695 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.202695 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:17.202695 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.202695 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.202695 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:17.202695 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.202695 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.203402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.204011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:17.204722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:17.205752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.207134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:17.207134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.207134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.207134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:17.207134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.207134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:17.208982 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:17.209494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:17.209614 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:17.210510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:17.210723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:17.210881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.211411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:17.211411 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.211411 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:17.211411 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.211411 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.211411 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:17.211411 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.211411 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.212932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.213420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:17.213889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.213889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.214636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.215262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:17.215898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.215898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:17.215898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:17.216649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:17.216907 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:17.217190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:17.217661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:17.217864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.218885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:52:17.218885 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.218885 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:17.218885 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.218885 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.218885 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:17.218885 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.218885 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:17.221069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:17.221928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.221928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:17.221928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.221928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:17.221928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.221928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.221928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:17.221928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.221928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.222595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:17.223097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.223599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.223599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:17.223599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.223599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:17.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.224937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:17.225616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.225616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:17.225616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.225616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.225616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:17.225616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.225616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:17.225616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:17.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:17.226482 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:17.226713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:17.227232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:17.227629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.228371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:52:17.228371 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.228371 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:17.228371 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.228371 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.228371 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:17.228371 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.228371 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:17.229748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:17.230351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.230351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:17.230351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.230351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:17.230351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.230351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.230351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:17.230351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.230351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:17.231328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.231328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:17.231328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:17.231328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:17.231328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.231328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:17.231328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:17.231328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:17.231328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:17.231328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.231328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:17.231767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.232253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.232253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:17.232253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.232253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.233199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:17.233836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.234570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.234570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:17.234570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.234570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:17.248571 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:17.248993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:17.249160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:17.249480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:17.249991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:17.250479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:17.250695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.251483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:52:17.251483 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.251483 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:17.251483 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.251483 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.251483 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:17.251483 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.251483 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.252839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.253683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:17.254484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:17.255365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:17.256087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:17.256087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:17.256087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:17.256087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:17.256087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:17.256087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:22.164432 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:22.165360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:22.165360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:22.166195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:22.201631 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:22.202141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.203199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:22.203199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.203199 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:22.203199 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.203199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.203199 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:22.203199 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.203199 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.204076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.204778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:22.205530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:22.206343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.207312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:22.207312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.207312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.207312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:22.207312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.207312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:22.209456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:22.209914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:22.210007 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:22.210969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:22.211390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:22.211646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.212451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:22.212451 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.212451 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:22.212451 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.212451 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.212451 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:22.212451 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.212451 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.213909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:22.214293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.214293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.214564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.215087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:22.215660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.215660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:22.216905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:22.217624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:22.217811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:22.218245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:22.218673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:22.219011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.219818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:52:22.219818 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.219818 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:22.219818 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.219818 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.219818 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:22.219818 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.219818 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:22.220881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:22.221486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.221486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:22.221486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.221486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:22.221486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.221486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.221486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:22.221486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.221486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.221813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:22.222246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.222696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.222696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:22.222696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.222696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:22.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.223293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:22.223824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.223824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:22.223824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.223824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.223824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:22.223824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.223824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:22.226063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:22.226910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:22.227116 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:22.227399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:22.228040 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:22.228589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.229655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:52:22.229655 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.229655 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:22.229655 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.229655 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.229655 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:22.229655 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.229655 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:22.231052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:22.231520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.231520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:22.231520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.231520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:22.231520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.231520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.231520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:22.231520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.231520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:22.232846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.232846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:22.232846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:22.232846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:22.232846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.232846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:22.232846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:22.232846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:22.232846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:22.232846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.232846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:22.233435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.234127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.234127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:22.234127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.234127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.234506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:22.235258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.236029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.236029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:22.236029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.236029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:22.249051 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:22.249812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:22.250012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:22.250366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:22.250948 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:22.251429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:22.251776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.252709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:52:22.252709 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.252709 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:22.252709 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.252709 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.252709 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:22.252709 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.252709 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.253717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.254379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:22.254957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:22.255735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:22.256996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:22.256996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:22.256996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:22.256996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:22.256996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:22.256996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:27.164901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:27.165601 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:27.165601 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:27.166498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:27.205191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:27.205431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.205998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:27.205998 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.205998 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:27.205998 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.205998 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.205998 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:27.205998 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.205998 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.206666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.207090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:27.207569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:27.208087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.208711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:27.208711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.208711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.208711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:27.208711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.208711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:27.209868 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:27.210633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:27.210712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:27.211426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:27.211625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:27.211784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.212543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:27.212543 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.212543 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:27.212543 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.212543 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.212543 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:27.212543 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.212543 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.213545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.213891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:27.214264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.214264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.214588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.215069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:27.215580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.215580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:27.217486 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:27.217942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:27.218027 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:27.218131 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:27.218518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:27.218721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.219261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:52:27.219261 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.219261 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:27.219261 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.219261 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.219261 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:27.219261 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.219261 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:27.219936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:27.220486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.220486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:27.220486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.220486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:27.220486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.220486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.220486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:27.220486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.220486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.220762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:27.221082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:27.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.221817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.222406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:27.226734 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:27.227864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:27.228123 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:27.229686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:27.229686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:27.229686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.229686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:52:27.229686 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.229686 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:27.229686 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.229686 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.229686 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:27.229686 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.229686 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:27.231030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:27.231519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.231519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:27.231519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.231519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:27.231519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.231519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.231519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:27.231519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.231519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:27.234030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.234030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:27.234030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:27.234030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:27.234030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.234030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:27.234030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:27.234030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:27.234030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:27.234030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.234030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:27.234437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:27.234863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:27.235317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.236033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:27.236033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:27.236033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.236033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:27.236033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.236033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:27.236033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.236033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.236033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:27.236033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.236033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:27.249682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:27.250192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:27.250350 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:27.250539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:27.251676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:27.251959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:27.252204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.252761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:52:27.252761 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.252761 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:27.252761 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.252761 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.252761 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:27.252761 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.252761 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.253997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.254609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:27.255253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.255253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:27.256679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:27.257226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:27.257715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:27.257715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:27.257715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:32.165614 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:32.166415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:32.166415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:32.167516 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:32.203135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:32.203624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.204690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:32.204690 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.204690 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:32.204690 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.204690 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.204690 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:32.204690 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.204690 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.205614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.206335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:32.206992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:32.207831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.208669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:32.208669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.208669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.208669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:32.208669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.208669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:32.210578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:32.211021 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:32.211174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:32.212664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:32.212897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:32.213058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.213627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:32.213627 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.213627 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:32.213627 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.213627 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.213627 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:32.213627 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.213627 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.215049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.215668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:32.216196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.216196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.216722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.217294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:32.217814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.217814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:32.217814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:32.218552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:32.218552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:32.218552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:32.218947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:32.219231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:52:32.220367 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.220367 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:32.220367 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.220367 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.220367 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:32.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.220367 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:32.220906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:32.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:32.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:32.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:32.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.221462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.221752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:32.222126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.222520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.222520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:32.222520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.222520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:32.222697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.223177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:32.223712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.223712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:32.223712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.223712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.223712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:32.223712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.223712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:32.228353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:32.228891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:32.228891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:32.229777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:32.230445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:32.230893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:52:32.231750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.231750 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:32.231750 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.231750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.231750 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:32.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:32.233024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:32.233801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.233801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:32.233801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.233801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:32.233801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.233801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.233801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:32.233801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.233801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:32.234443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.234443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:32.234443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:32.234443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:32.234443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.234443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:32.234443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:32.234443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:32.234443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:32.234443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.234443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:32.234689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.234944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.234944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:32.234944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.234944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.236401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:32.236850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.237276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.237276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:32.237276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.237276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:32.250074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:32.250499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:32.250693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:32.250977 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:32.251557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:32.252078 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:32.252389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.253299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Traceback (most recent call last):
Apr 21 13:52:32.253299 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.253299 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:32.253299 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.253299 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.253299 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:32.253299 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.253299 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.254744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:32.255931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.255931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:32.256484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:32.256937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:32.257434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:32.257434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:32.257434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:37.166638 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:37.167474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:37.167590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:37.168837 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:37.211358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:37.212770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:37.212915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:37.213423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:37.213866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.215231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:37.215231 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.215231 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:37.215231 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.215231 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.215231 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:37.215231 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.215231 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.216494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.217124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:37.217935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:37.218702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:37.219291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.219291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:37.219291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.219291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:37.219291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.219291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.219291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:37.219291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.219291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:37.219291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:37.219291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.220404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:37.220639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:37.220970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:37.220970 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.220970 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:37.220970 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.220970 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.220970 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:37.220970 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.220970 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:37.221379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.222445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.222893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:37.223284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.223284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:37.223284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:37.223284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.224345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.224906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:37.225552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.225552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:37.225552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.225552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:52:37.225552 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.225552 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:37.225552 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.225552 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.225552 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:37.225552 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.225552 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:37.226919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:37.227437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.227437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:37.227437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.227437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:37.227437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.227437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.227437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:37.227437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.227437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.228812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:37.229407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.230307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.230307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:37.230307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.230307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:37.230307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:37.231913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.232510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:37.232956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.232956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:37.232956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.232956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.232956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:37.232956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.232956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:37.232956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:37.232956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:37.232956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:37.233577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:37.233983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.234811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:52:37.234811 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.234811 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:37.234811 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.234811 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.234811 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:37.234811 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.234811 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:37.235648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:37.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:37.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:37.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:37.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:37.236669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.236669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:37.236669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:37.236669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:37.236669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.236669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:37.236669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:37.236669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:37.236669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:37.236669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.236669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:37.236983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.237394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.237394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:37.237394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.237394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.237784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:37.238266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.238823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.238823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:37.238823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.238823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:37.250656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:37.251600 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:37.251800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:37.251970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:37.252528 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:37.253053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:37.253494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.254356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:52:37.254356 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.254356 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:37.254356 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.254356 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.254356 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:37.254356 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.254356 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.255304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.255685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:37.256101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.256101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:37.256591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:37.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:37.258025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:37.258025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:37.258025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:42.167386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:42.169022 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:42.169022 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:42.169850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:42.171479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:52:42.201689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:42.201995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:42.202793 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.202793 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:42.202793 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.202793 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.202793 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:42.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.202793 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.203896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.204452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:42.205179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.205179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:42.205341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.206020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.206896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:42.206896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.206896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:42.212695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:42.213086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:42.213166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:42.214228 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:42.214496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:42.214656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.215305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:42.215305 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.215305 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:42.215305 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.215305 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.215305 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:42.215305 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.215305 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.216274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.216867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:42.217352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.217352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.217461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.218164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:42.218819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.218819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:42.220263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:42.220642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:42.220774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:42.220904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:42.221625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:42.221854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.222451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:52:42.222451 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.222451 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:42.222451 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.222451 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.222451 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:42.222451 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.222451 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:42.223018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:42.223501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.223501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:42.223501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.223501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:42.223501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.223501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.223501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:42.223501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.223501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.223790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:42.224139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:42.224578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.224958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.225514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:42.230227 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:42.231120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:42.231120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:42.231120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:42.231549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:42.231832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.234341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:42.234969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.236038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:42.236659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:42.237786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.239526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:42.239526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.239526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:42.239526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.239526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.239526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:42.239526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.239526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:42.251245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:42.252387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:42.252387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:42.252387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:42.252765 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:42.253051 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:42.253298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.254281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:52:42.254281 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.254281 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:42.254281 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.254281 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.254281 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:42.254281 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.254281 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.255368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:42.256697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:42.257470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:42.258542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:42.258542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:42.258542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:42.258542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:42.258542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:42.258542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:47.168360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:47.169335 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:47.169335 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:47.170856 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:47.208492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:47.208492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.209248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:47.209248 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.209248 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:47.209248 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.209248 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.209248 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:47.209248 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.209248 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.210571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.211495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:47.212146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:47.213017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.215664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:47.215664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.215664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.215664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:47.215664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.215664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:47.215664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:47.215664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:47.215664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:47.215664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:47.217161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:47.218271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:47.218877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.219541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.220300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:47.220719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:47.221428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:47.221926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:47.222134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:47.222590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:47.222788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.223440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:52:47.223440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.223440 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:47.223440 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.223440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.223440 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:47.223440 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.223440 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:47.225014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:47.225423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.225423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:47.225423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.225423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:47.225423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.225423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.225423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:47.225423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.225423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.226192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:47.226681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.227045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.227045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:47.227045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.227045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:47.228126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.228623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:47.229105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.229105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:47.229105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.229105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.229105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:47.229105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.229105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:47.230714 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:47.231155 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:47.231280 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:47.231367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:47.231751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:47.231979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.232526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:52:47.232526 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.232526 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:47.232526 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.232526 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.232526 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:47.232526 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.232526 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:47.233195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:47.233596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.233596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:47.233596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.233596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:47.233596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.233596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.233596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:47.233596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.233596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:47.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:47.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:47.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:47.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:47.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:47.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:47.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:47.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:47.234136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.234531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.234531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:47.234531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.234531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.234646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:47.235539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.237261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.237261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:47.237261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.237261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:47.251889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:47.252536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:47.252536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:47.252536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:47.253057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:47.253528 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:47.253757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.254497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:52:47.254497 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.254497 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:47.254497 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.254497 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.254497 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:47.254497 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.254497 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.255166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.255953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:47.256575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:47.257494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:47.261293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:47.261293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:47.261293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:47.261293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:47.261293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:47.261293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:52.169204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:52.170558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:52.170558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:52.171053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:52.208144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:52.208589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.209439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:52.209439 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.209439 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:52.209439 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.209439 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.209439 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:52.209439 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.209439 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.210233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.210581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:52.210926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.210926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:52.210987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.211399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.211846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:52.211846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.211846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:52.213534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:52.213905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:52.214055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:52.214881 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:52.215330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:52.215670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.217567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:52.217567 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.217567 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:52.217567 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.217567 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.217567 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:52.217567 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.217567 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.219051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.219335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:52.219563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.219563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.222666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.223049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:52.223409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.223409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:52.223409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:52.223512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:52.223576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:52.223647 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:52.224567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:52.224923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.225769 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:52.226356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.226831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:52.226831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.226831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:52.226831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.226831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.226831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:52.226831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.226831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.227066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:52.227412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.227744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:52.228106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.228611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:52.228611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:52.228611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.228611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:52.228611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.228611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:52.228611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.228611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.228611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:52.228611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.228611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:52.231711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:52.232167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:52.232273 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:52.232398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:52.232864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:52.233153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.233767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:52:52.233767 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.233767 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:52.233767 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.233767 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.233767 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:52.233767 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.233767 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:52.234476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:52.234985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.234985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:52.234985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.234985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:52.234985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.234985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.234985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:52.234985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.234985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:52.235277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.235277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:52.235277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:52.235277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:52.235277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.235277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:52.235277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:52.235277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:52.235277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:52.235277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.235277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:52.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:52.236179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:52.236633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.237129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:52.237129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:52.237129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.237129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:52.237129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.237129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:52.237129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.237129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.237129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:52.237129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.237129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:52.252645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:52.253684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:52.253684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:52.253684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:52.254430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:52.254850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:52.255127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:52:52.256027 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.256027 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:52.256027 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.256027 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.256027 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:52.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.256027 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.256956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.257488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:52.258000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.258000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:52.258197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:52.258821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:52.259519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:52.259519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:52.259519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:57.170447 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:57.170447 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:57.171469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:57.172021 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:57.211294 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:57.211783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.212753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:52:57.212753 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.212753 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:57.212753 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.212753 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.212753 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:57.212753 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.212753 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.213775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.214410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:57.214970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:57.215814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.216656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:57.216656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.216656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.216656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:57.216656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.216656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:57.216656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:57.216656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:57.216656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:57.217157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:57.217157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:57.217157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.218581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:57.218581 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.218581 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:57.218581 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.218581 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.218581 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:57.218581 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.218581 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.220134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.220567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:57.220957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.220957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.221917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.222482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:57.223026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.223026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:57.223644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:57.223976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:57.224111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:57.224272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:52:57.224699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:57.224899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.225479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:52:57.225479 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.225479 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:57.225479 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.225479 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.225479 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:57.225479 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.225479 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:57.226410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:57.227044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.227044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:57.227044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.227044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:57.227044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.227044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.227044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:57.227044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.227044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.227537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:57.227978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:57.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:57.228867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.229597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:57.230253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.230253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:57.230253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.230253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.230253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:57.230253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.230253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:57.264401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:57.264564 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:52:57.264917 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:57.265966 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:52:57.266112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:57.266293 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:52:57.266482 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:57.266628 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:52:57.267517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:57.267964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.269014 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:57.269417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:52:57.269669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.272033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:52:57.272033 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.272033 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:57.272033 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.272033 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.272033 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:57.272033 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.272033 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:57.272519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:52:57.272519 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.272519 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:52:57.272519 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.272519 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.272519 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:52:57.272519 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.272519 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:52:57.274122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_mac_aging_settings failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:57.274957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:57.275641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:57.276179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.276746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.277606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:57.279815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.281464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:52:57.281464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.281464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:52:57.281644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:52:57.282576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:52:57.283402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:52:57.283402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:52:57.283402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:52:57.283402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:02.171509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:02.172556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:02.172556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:02.172769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:02.218718 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:02.219123 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:02.219217 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:02.220422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:02.220688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:02.220897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.222363 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:02.222667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.223459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:53:02.223459 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.223459 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:02.223459 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.223459 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.223459 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:02.223459 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.223459 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:02.224462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:02.224462 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.224462 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:02.224462 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.224462 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.224462 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:02.224462 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.224462 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.226285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.226592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:02.226881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.226881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.228241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.229069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:02.229663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.229663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:02.230184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.231047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.232448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:02.233450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.234460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.234460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:02.234460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.234460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:02.234460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:02.234460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:02.234460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:02.234460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:02.234886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:02.234886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.235004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:53:02.235004 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.235004 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:02.235004 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.235004 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.235004 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:02.235004 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.235004 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:02.235951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:02.236668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.236668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:02.236668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.236668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:02.236668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.236668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.236668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:02.236668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.236668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.236975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:02.237347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.237712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.237712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:02.237712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.237712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:02.237831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.238327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:02.238805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.238805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:02.238805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.238805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.238805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:02.238805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.238805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:02.265719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:02.265719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:02.266097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:02.266449 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:02.267295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:02.267446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:02.267725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:02.268437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:02.268701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:02.269042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:02.269241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.269969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:02.270187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.270838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:53:02.270838 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.270838 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:02.270838 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.270838 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.270838 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:02.270838 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.270838 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.273029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.273454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:02.273802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.273802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:02.274189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.274795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.275547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:02.275547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.275547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:02.275547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:02.275547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.275547 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:02.275547 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.275547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.275547 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:02.275547 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.275547 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:02.276191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.276786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:02.276786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.276786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:02.276786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.276786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.276786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:02.276786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.276786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:02.277143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.277549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.277923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:02.277923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.277923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:02.278139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:02.278631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:02.279147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:02.279147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:02.279147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:02.279147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:07.171769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:07.172778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:07.172778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:07.174021 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:07.215581 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:07.215948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.216789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:07.216789 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.216789 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:07.216789 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.216789 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.216789 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:07.216789 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.216789 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.217737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.218765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:07.219401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:07.220188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.221092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:07.221092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.221092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.221092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:07.221092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.221092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:07.221092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:07.221092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:07.221092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:07.221092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:07.222778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:07.222778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.222778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:07.222778 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.222778 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:07.222778 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.222778 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.222778 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:07.222778 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.222778 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.223896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.224883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:07.225689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.225689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.226558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.227580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:07.228266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.228266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:07.232474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:07.233467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:07.233822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:07.234242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:07.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:07.235096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.235922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:53:07.235922 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.235922 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:07.235922 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.235922 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.235922 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:07.235922 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.235922 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:07.237525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:07.237999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.237999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:07.237999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.237999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:07.237999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.237999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.237999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:07.237999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.237999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.239190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:07.239998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:07.240763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.241334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.241846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:07.266325 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:07.267253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:07.268118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:07.269596 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:07.270152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:07.270629 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:07.270999 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:07.271458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:07.272031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:07.272516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.273386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:07.273860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:07.273860 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.273860 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:07.273860 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.273860 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.273860 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:07.273860 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.273860 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:07.274452 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:07.275335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.276068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:07.276068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.276068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:07.276068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.276068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.276068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:07.276068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.276068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:07.276068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:07.277176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.277732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.278269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:07.278269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.278269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:07.278269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Traceback (most recent call last):
Apr 21 13:53:07.278269 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.278269 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:07.278269 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.278269 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.278269 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:07.278269 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.278269 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.279202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:07.280688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.281711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:07.282229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.282563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.282563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:07.282563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.282563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:07.282861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:07.283321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:07.283810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:07.283810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:07.283810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:12.172861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:12.173792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:12.173988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:12.175262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:12.211339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:12.211673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.213338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:12.213338 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.213338 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:12.213338 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.213338 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.213338 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:12.213338 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.213338 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.213683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.214096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:12.214534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:12.215012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.215629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:12.215629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.215629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.215629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:12.215629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.215629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:12.220168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:12.220636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:12.220919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:12.221660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:12.221942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:12.222169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.222876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:12.222876 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.222876 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:12.222876 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.222876 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.222876 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:12.222876 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.222876 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.224404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.224826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:12.225146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.225146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.226075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.226999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:12.227585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.227585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:12.233100 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:12.233830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:12.234019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:12.234300 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:12.234845 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:12.235122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.236161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:53:12.236161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.236161 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:12.236161 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.236161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.236161 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:12.236161 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.236161 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:12.237641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:12.238565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.238565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:12.238565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.238565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:12.238565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.238565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.238565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:12.238565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.238565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.239159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:12.239906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.240839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.240839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:12.240839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.240839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:12.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.241768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:12.242603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.242603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:12.242603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.242603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.242603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:12.242603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.242603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:12.268146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:12.268401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:12.269365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:12.269823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:12.270191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:12.270634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:12.271137 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:12.271903 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:12.272308 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:12.272720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.273583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:53:12.273583 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.273583 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:12.273583 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.273583 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.273583 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:12.273583 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.273583 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:12.275155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:12.275984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.275984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:12.275984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.275984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:12.275984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.275984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.275984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:12.275984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.275984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:12.275984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:12.277196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.277196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:12.277196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:12.277196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:12.277196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.277196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:12.277196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:12.277196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:12.277196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:12.277196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.277196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:12.278030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.278702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.278702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:12.278702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.278702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:12.278702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.279975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:12.280967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.281964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.281964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:12.281964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.281964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:12.282745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.283387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:53:12.283387 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.283387 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:12.283387 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.283387 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.283387 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:12.283387 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.283387 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task update_mac_aging_settings failed after 0.013 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.285170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:12.285540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.285540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:12.286230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:12.286768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:12.287291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:12.287291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:12.287291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:17.173775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:17.174299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:17.174428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:17.175679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:17.212781 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:17.213453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.213882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:17.213882 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.213882 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:17.213882 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.213882 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.213882 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:17.213882 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.213882 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.214956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.215617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:17.216080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:17.216720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.217488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:17.217488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.217488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.217488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:17.217488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.217488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:17.220599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:17.221196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:17.221348 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:17.222951 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:17.223245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:17.223400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.223964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:17.223964 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.223964 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:17.223964 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.223964 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.223964 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:17.223964 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.223964 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.225728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.226460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:17.227040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.227040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.227535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.228553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:17.229593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.229593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:17.233631 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:17.234360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:17.234561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:17.234727 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:17.235151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:17.235428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.236105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:53:17.236105 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.236105 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:17.236105 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.236105 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.236105 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:17.236105 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.236105 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:17.238392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:17.238920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.238920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:17.238920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.238920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:17.238920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.238920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.238920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:17.238920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.238920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.240385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:17.240754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:17.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:17.242699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.243134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:17.243630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.243630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:17.243630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.243630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.243630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:17.243630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.243630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:17.269193 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:17.269503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:17.270342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:17.270565 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:17.270862 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:17.271961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:17.271961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:17.272279 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:17.273075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:17.273581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.274340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:17.274571 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:17.274730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.275223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:53:17.275223 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.275223 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:17.275223 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.275223 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.275223 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:17.275223 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.275223 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:17.277171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.278088 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.279093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.279878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:17.281069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.281069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:17.281069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.281069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:17.281069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:17.281069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:17.281069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.281069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:17.281069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:17.281069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:17.281069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:17.281649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.282053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.282452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.282976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:17.283316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:17.283741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:17.284261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:17.284261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:17.284261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:22.174588 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:22.175496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:22.175496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:22.176559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:22.210080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:22.210418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.210923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:22.210923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.210923 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:22.210923 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.210923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.210923 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:22.210923 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.210923 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.211585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.212276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:22.212943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:22.213764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.214902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:22.214902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.214902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.214902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:22.214902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.214902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:22.220940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:22.221528 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:22.221680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:22.222651 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:22.222967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:22.223235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.223959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:22.223959 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.223959 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:22.223959 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.223959 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.223959 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:22.223959 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.223959 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.225505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.225904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:22.226389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.226389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.227127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.227944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:22.228875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.228875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:22.234249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:22.234901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:22.235089 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:22.235343 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:22.235818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:22.236087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.236896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:53:22.236896 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.236896 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:22.236896 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.236896 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.236896 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:22.236896 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.236896 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:22.238124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:22.238827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.238827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:22.238827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.238827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:22.238827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.238827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.238827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:22.238827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.238827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.239467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:22.240095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.240802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.240802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:22.240802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.240802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:22.241062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.241822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:22.242772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.242772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:22.242772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.242772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.242772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:22.242772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.242772 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:22.270019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:22.270723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:22.270900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:22.271142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:22.271806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:22.272388 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:22.272756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.274108 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:22.274421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:22.275189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:53:22.275189 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.275189 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:22.275189 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.275189 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.275189 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:22.275189 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.275189 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:22.275622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:22.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:22.278575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.278575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:22.278575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.278575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:22.278575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.278575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.278575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:22.278575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.278575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:22.278575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:22.279808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.279808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:22.279808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:22.279808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:22.279808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.279808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:22.279808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:22.279808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:22.279808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:22.279808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.279808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:22.280412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.281246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.281246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:22.281246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.281246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:22.281246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.281666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:22.282542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.283240 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_mac_aging_settings failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.284300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.284668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:22.285023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.285023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:22.285508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:22.285974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:22.286642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:22.286642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:22.286642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:27.175344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:27.176444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:27.176444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:27.176845 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:27.216843 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:27.217509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.218607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:27.218607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.218607 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:27.218607 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.218607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.218607 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:27.218607 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.218607 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.219625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.220355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:27.221017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:27.221838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:27.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:27.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:27.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:27.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:27.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:27.222817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:27.223386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:27.223386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.224242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:27.224242 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.224242 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:27.224242 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.224242 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.224242 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:27.224242 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.224242 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.226126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.226553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:27.226911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.226911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.228385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.228898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:27.229456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.229456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:27.234880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:27.235423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:27.235548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:27.235680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:27.236141 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:27.236460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.237234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:53:27.237234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.237234 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:27.237234 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.237234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.237234 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:27.237234 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.237234 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:27.238681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:27.239490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.239490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:27.239490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.239490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:27.239490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.239490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.239490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:27.239490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.239490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.240157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:27.240812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.241446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.241446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:27.241446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.241446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:27.242434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.243148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:27.244324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.244324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:27.244324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.244324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.244324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:27.244324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.244324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:27.270638 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:27.271193 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:27.271413 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:27.271621 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:27.272242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:27.272625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.273439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:53:27.273439 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.273439 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:27.273439 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.273439 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.273439 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:27.273439 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.273439 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:27.274842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:27.275287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.275287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:27.275287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.275287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:27.275287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.275287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.275287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:27.275287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.275287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:27.275287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:27.277294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.277294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:27.277294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:27.277294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:27.277294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.277294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:27.277294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:27.277294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:27.277294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:27.277294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.277294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:27.277688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.278073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.278073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:27.278073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.278073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:27.278073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:27.279744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.280344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.280344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:27.280344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.280344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:27.280344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:27.280344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:27.280708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:27.281052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:27.281313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.282092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:53:27.282092 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.282092 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:27.282092 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.282092 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.282092 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:27.282092 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.282092 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.283622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.284057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:27.284452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.284452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:27.285295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:27.285767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:27.286329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:27.286329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:27.286329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:32.176283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:32.177940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:32.178123 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:32.179800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:32.219928 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:32.220189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.220975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:32.220975 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.220975 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:32.220975 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.220975 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.220975 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:32.220975 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.220975 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.222853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.224119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:32.225150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:32.225924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.226901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:32.226901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.226901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.226901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:32.226901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.226901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:32.226901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:32.226901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:32.226901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:32.226901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:32.227520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:32.228190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:32.228963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.229669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.230304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:32.235538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:32.236592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:32.236818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:32.237035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:32.237645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:32.237937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.238835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:53:32.238835 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.238835 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:32.238835 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.238835 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.238835 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:32.238835 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.238835 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:32.240329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:32.241572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.241572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:32.241572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.241572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:32.241572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.241572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.241572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:32.241572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.241572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.242280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:32.242923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:32.243669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.245366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.246694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:32.271841 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:32.272020 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:32.274387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:32.274557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:32.275438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:32.275687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.276338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:53:32.276338 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.276338 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:32.276338 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.276338 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.276338 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:32.276338 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.276338 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:32.276959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:32.277698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:32.278502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.278502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:32.278502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.278502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:32.278502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.278502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.278502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:32.278502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.278502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:32.279110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.279110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:32.279110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:32.279110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:32.279110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.279110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:32.279110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:32.279110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:32.279110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:32.279110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.279110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:32.279819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:32.280531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:32.281386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:32.282161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.282560 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.282943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.283282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:32.283630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:32.284049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:32.284722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:32.284722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:32.284722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:32.284722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:32.284722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:32.284722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:37.178066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:37.178955 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:37.179053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:37.180260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:37.212516 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:37.212923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.213941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:37.213941 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.213941 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:37.213941 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.213941 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.213941 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:37.213941 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.213941 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.214812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.215740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:37.216893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:37.217750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.219064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:37.219064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.219064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.219064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:37.219064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.219064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:37.224087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:37.224641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:37.224794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:37.225923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:37.226298 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:37.226600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.227646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:37.227646 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.227646 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:37.227646 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.227646 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.227646 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:37.227646 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.227646 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.228918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.229321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:37.229765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.229765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.230393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.230945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:37.231638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.231638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:37.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:37.236864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:37.237096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:37.237420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:37.237869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:37.238085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.238808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:53:37.238808 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.238808 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:37.238808 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.238808 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.238808 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:37.238808 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.238808 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:37.240172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:37.240933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.240933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:37.240933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.240933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:37.240933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.240933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.240933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:37.240933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.240933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.241686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:37.242325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.243020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.243020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:37.243020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.243020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:37.243402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.244281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:37.245081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.245081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:37.245081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.245081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.245081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:37.245081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.245081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:37.272131 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:37.273357 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:37.273500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:37.273596 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:37.274244 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:37.274583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.275181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:53:37.275181 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.275181 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:37.275181 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.275181 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.275181 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:37.275181 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.275181 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:37.275930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:37.276769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.276769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:37.276769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.276769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:37.276769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.276769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.276769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:37.276769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.276769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:37.277102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.277102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:37.277102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:37.277102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:37.277102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.277102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:37.277102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:37.277102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:37.277102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:37.277102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.277102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:37.277717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:37.278114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:37.278594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.279167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:37.279167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:37.279167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.279167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:37.279167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.279167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:37.279167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.279167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.279167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:37.279167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.279167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:37.280031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:37.280363 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:37.280468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:37.280615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:37.280948 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:37.281177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:37.281353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.281917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:53:37.281917 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.281917 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:37.281917 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.281917 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.281917 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:37.281917 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.281917 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.282857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.283414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:37.283807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.283807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:37.283903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:37.284425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:37.285004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:37.285004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:37.285004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:42.178810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:42.179402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:42.179402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:42.180115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:42.181664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:53:42.211429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:42.211707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.212482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:42.212482 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.212482 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:42.212482 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.212482 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.212482 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:42.212482 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.212482 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.213445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.214021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:42.214780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:42.215620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.216631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:42.216631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.216631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.216631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:42.216631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.216631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:42.224789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:42.225525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:42.225793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:42.226800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:42.227119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:42.227253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.227856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:42.227856 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.227856 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:42.227856 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.227856 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.227856 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:42.227856 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.227856 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.229598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.230270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:42.230818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.230818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.231364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.232237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:42.233119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.233119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:42.236796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:42.237234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:42.237385 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:42.237524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:42.238140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:42.238478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.239323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:53:42.239323 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.239323 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:42.239323 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.239323 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.239323 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:42.239323 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.239323 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:42.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:42.240974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.240974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:42.240974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.240974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:42.240974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.240974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.240974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:42.240974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.240974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.241365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:42.241732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.242152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.242152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:42.242152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.242152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:42.243070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.243637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:42.244190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.244190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:42.244190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.244190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.244190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:42.244190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.244190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:42.273417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:42.274465 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:42.274649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:42.274861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:42.275663 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:42.276150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.276943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:53:42.276943 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.276943 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:42.276943 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.276943 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.276943 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:42.276943 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.276943 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:42.278391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:42.279266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.279266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:42.279266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.279266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:42.279266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.279266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.279266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:42.279266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.279266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:42.280125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.280125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:42.280125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:42.280125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:42.280125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.280125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:42.280125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:42.280125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:42.280125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:42.280125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.280125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:42.280685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.281064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.281064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:42.281064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.281064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.282142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:42.282704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.283260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.283260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:42.283260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.283260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:42.283260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:42.283260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:42.283260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:42.283260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:42.283833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:42.284116 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:42.284292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.284985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:53:42.284985 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.284985 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:42.284985 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.284985 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.284985 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:42.284985 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.284985 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.285863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.286224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:42.286561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.286561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:42.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:42.287510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:42.288082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:42.288082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:42.288082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:47.179566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:47.180820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:47.180820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:47.181673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:47.225541 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:47.226530 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:47.226530 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:47.227187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:47.227574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.228263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:47.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:47.228603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.228984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:47.228984 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.228984 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:47.228984 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.228984 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.228984 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:47.228984 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.228984 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.230124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.231162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:47.232198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:47.233053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:47.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:47.234885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:47.235312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.235735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.236377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:47.237478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:47.237901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:47.238061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:47.238278 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:47.238659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:47.238884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.239442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:53:47.239442 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.239442 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:47.239442 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.239442 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.239442 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:47.239442 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.239442 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:47.240397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:47.240915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.240915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:47.240915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.240915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:47.240915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.240915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.240915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:47.240915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.240915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.241563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:47.241994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.242501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.242501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:47.242501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.242501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:47.243011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.243514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:47.244067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.244067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:47.244067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.244067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.244067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:47.244067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.244067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:47.274484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:47.275320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:47.275514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:47.275786 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:47.276410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:47.276840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.277780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:53:47.277780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.277780 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:47.277780 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.277780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.277780 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:47.277780 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.277780 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:47.278890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:47.279602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.279602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:47.279602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.279602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:47.279602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.279602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.279602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:47.279602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.279602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:47.280062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.280062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:47.280062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:47.280062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:47.280062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.280062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:47.280062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:47.280062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:47.280062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:47.280062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.280062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:47.280606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:47.281193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:47.281956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:47.282870 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:47.283549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:47.283549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:47.283752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:47.284154 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:47.284430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:47.284559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.285078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:53:47.285078 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.285078 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:47.285078 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.285078 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.285078 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:47.285078 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.285078 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.286180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.286773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:47.287146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.287146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:47.287553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:47.288029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:47.288763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:47.288763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:47.288763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:52.180191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:52.181305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:52.181305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:52.182324 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:52.219838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:52.220298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.221250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:52.221250 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.221250 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:52.221250 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.221250 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.221250 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:52.221250 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.221250 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.222301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.223121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:52.223848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:52.224661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.225865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:52.225865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.225865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.225865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:52.225865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.225865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:52.226527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:52.227140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:52.227395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:52.228439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:52.228795 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:52.229133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.230007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:52.230007 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.230007 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:52.230007 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.230007 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.230007 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:52.230007 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.230007 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.231796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.232231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:52.232756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.232756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.233534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.234059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:52.234660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.234660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:52.237923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:52.238499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:52.238725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:52.238964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:52.239542 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:52.239836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.240878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:53:52.240878 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.240878 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:52.240878 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.240878 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.240878 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:52.240878 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.240878 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:52.242019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:52.242852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.242852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:52.242852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.242852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:52.242852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.242852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.242852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:52.242852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.242852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.243374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:52.243940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:52.244555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.245199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.246010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:52.275204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:52.275760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:52.275948 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:52.276150 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:52.277082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:52.277666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.278675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:53:52.278675 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.278675 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:52.278675 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.278675 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.278675 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:52.278675 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.278675 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:52.280121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:52.280611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.280611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:52.280611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.280611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:52.280611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.280611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.280611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:52.280611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.280611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:52.281560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.281560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:52.281560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:52.281560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:52.281560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.281560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:52.281560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:52.281560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:52.281560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:52.281560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.281560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:52.281943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.282461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.282461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:52.282461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.282461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.283862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:52.284497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.285064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.285064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:52.285064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.285064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:52.285064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:52.285064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:52.285064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:52.285064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:52.285548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:52.285951 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:52.286204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.287100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:53:52.287100 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.287100 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:52.287100 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.287100 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.287100 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:52.287100 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.287100 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.288065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.288564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:52.288978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.288978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:52.289188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:52.289732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:52.290332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:52.290332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:52.290332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:57.180872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:57.181950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:57.181950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:57.182993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:57.217088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:57.217505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.218290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:53:57.218290 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.218290 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:57.218290 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.218290 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.218290 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:57.218290 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.218290 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.219357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.220041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:57.220843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:57.221904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.223156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:57.223156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.223156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.223156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:57.223156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.223156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:57.227396 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:57.228033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:57.228167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:57.229122 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:57.229539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:57.229816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.230680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:53:57.230680 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.230680 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:57.230680 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.230680 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.230680 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:57.230680 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.230680 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.233381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:57.233719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.233719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.236015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.236834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:57.237695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.237695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:57.238718 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:57.239228 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:57.239370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:57.239503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:53:57.240090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:57.240448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.241200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:53:57.241200 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.241200 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:57.241200 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.241200 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.241200 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:57.241200 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.241200 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:57.242292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:57.242968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.242968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:57.242968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.242968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:57.242968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.242968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.242968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:57.242968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.242968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.243301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:57.243667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.244038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.244038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:57.244038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.244038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:57.244156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.244642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:57.245141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.245141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:57.245141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.245141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.245141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:57.245141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.245141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:57.275794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:57.276462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:57.276717 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:57.276979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:57.277742 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:57.278286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.279397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:53:57.279397 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.279397 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:57.279397 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.279397 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.279397 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:57.279397 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.279397 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:53:57.281498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:53:57.282495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.282495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:53:57.282495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.282495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:53:57.282495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.282495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.282495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:53:57.282495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.282495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:53:57.284021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.284021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:57.284021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:57.284021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:57.284021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.284021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:57.284021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:57.284021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:57.284021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:57.284021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.284021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:57.284686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.285360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.285360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:57.285360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.285360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.285944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:57.286808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.287387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.287387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:57.287387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.287387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:53:57.287387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:53:57.287387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:53:57.287387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:53:57.287387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:53:57.287699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:57.287900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:53:57.288038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.288497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:53:57.288497 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.288497 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:53:57.288497 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.288497 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.288497 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:53:57.288497 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.288497 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.289134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:53:57.289885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.289885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:53:57.290262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:53:57.290815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:53:57.291464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:53:57.291464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:53:57.291464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:02.181679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:02.182810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:02.182810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:02.183742 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:02.220951 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:02.221451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.222538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:02.222538 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.222538 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:02.222538 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.222538 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.222538 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:02.222538 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.222538 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.223494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.224109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:02.224582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:02.225063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.225681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:02.225681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.225681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.225681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:02.225681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.225681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:02.228069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:02.228276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:02.228346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:02.229160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:02.229599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:02.229867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.230883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:54:02.230883 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.230883 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:02.230883 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.230883 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.230883 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:02.230883 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.230883 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.231881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.232280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:02.232733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.232733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.233456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.233937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:02.234491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.234491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:02.239246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:02.239990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:02.240194 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:02.240484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:02.241371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:02.241809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.243414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:54:02.243414 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.243414 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:02.243414 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.243414 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.243414 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:02.243414 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.243414 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:02.244806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:02.245625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.245625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:02.245625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.245625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:02.245625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.245625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.245625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:02.245625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.245625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.246741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:02.247608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.248104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.248104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:02.248104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.248104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:02.248410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.248909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:02.249593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.249593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:02.249593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.249593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.249593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:02.249593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.249593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:02.276435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:02.277039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:02.277327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:02.277809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:02.278835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:02.279432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.280479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:54:02.280479 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.280479 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:02.280479 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.280479 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.280479 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:02.280479 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.280479 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:02.281819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:02.282536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.282536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:02.282536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.282536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:02.282536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.282536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.282536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:02.282536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.282536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:02.283561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.283561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:02.283561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:02.283561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:02.283561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.283561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:02.283561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:02.283561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:02.283561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:02.283561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.283561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:02.284201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.284836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.284836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:02.284836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.284836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.285689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:02.286703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.287815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.287815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:02.287815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.287815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:02.287815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:02.287815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:02.287815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:02.288298 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:02.288565 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:02.288945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:02.289242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.290089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:54:02.290089 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.290089 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:02.290089 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.290089 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.290089 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:02.290089 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.290089 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.291012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.291758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:02.292173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:02.292607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:02.293116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:02.293116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:02.293116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:02.293116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:02.293116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:02.293116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:07.182810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:07.183961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:07.184173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:07.185400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:07.218357 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:07.218728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.219723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:07.219723 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.219723 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:07.219723 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.219723 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.219723 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:07.219723 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.219723 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.220692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.221268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:07.221735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:07.222243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.222807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:07.222807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.222807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.222807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:07.222807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.222807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:07.228525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:07.228819 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:07.228986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:07.229814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:07.230029 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:07.230345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.231091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:54:07.231091 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.231091 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:07.231091 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.231091 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.231091 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:07.231091 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.231091 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.232789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.233357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:07.233792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.233792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.234628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.235159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:07.235832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.235832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:07.239733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:07.240515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:07.240667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:07.240830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:07.241646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:07.241927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.242903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:54:07.242903 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.242903 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:07.242903 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.242903 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.242903 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:07.242903 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.242903 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:07.244388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:07.244780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.244780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:07.244780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.244780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:07.244780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.244780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.244780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:07.244780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.244780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.246065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:07.247488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:07.248276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.249075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.250252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:07.277190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:07.278113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:07.278390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:07.278590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:07.279638 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:07.280296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.281523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:54:07.281523 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.281523 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:07.281523 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.281523 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.281523 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:07.281523 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.281523 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:07.283187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:07.284004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.284004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:07.284004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.284004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:07.284004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.284004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.284004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:07.284004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.284004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:07.284474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.284474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:07.284474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:07.284474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:07.284474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.284474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:07.284474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:07.284474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:07.284474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:07.284474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.284474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:07.284995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:07.285715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:07.286818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:07.287993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:07.288892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:07.288892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:07.289007 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:07.289445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:07.289735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.290586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:54:07.290586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.290586 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:07.290586 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.290586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.290586 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:07.290586 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.290586 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.292114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.292405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:07.292653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.292653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:07.293115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:07.293728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:07.294143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:07.294143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:07.294143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:12.184377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:12.185046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:12.185298 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:12.186708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:12.229463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:12.230603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:12.232006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:12.232233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.233139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:12.234086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:12.234086 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.234086 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:12.234086 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.234086 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.234086 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:12.234086 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.234086 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:12.234679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.236348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.236909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:12.237401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.237401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:12.237401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:12.238153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.238921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.239586 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.240593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.241040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:12.241444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.241444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.241740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.242326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:12.242899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.242899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:12.242899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:12.243015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:12.243142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:12.243312 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:12.243891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:12.244164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.244835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:54:12.244835 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.244835 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:12.244835 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.244835 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.244835 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:12.244835 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.244835 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:12.246021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:12.246606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.246606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:12.246606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.246606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:12.246606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.246606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.246606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:12.246606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.246606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.247168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:12.247558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.247930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.247930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:12.247930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.247930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:12.248705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.249149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:12.249633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.249633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:12.249633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.249633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.249633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:12.249633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.249633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:12.278248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:12.279041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:12.279348 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:12.279665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:12.280511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:12.280947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.281993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:54:12.281993 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.281993 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:12.281993 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.281993 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.281993 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:12.281993 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.281993 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:12.283780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:12.284542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.284542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:12.284542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.284542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:12.284542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.284542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.284542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:12.284542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.284542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:12.286011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.286011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:12.286011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:12.286011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:12.286011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.286011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:12.286011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:12.286011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:12.286011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:12.286011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.286011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:12.286606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.287260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.287260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:12.287260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.287260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.287781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:12.288584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.289443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.289443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:12.289443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.289443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:12.289443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:12.289443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:12.289443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:12.289825 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:12.290320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:12.290670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:12.291323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.291934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:54:12.291934 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.291934 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:12.291934 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.291934 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.291934 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:12.291934 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.291934 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.293555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.293955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:12.294336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.294336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:12.295874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:12.296933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:12.297797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:12.297797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:12.297797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:17.185056 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:17.185997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:17.185997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:17.187132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:17.226483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:17.226819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.227540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:17.227540 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.227540 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:17.227540 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.227540 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.227540 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:17.227540 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.227540 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.228312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.229094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:17.229624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:17.230118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.230835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:17.230835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.230835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.230835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:17.230835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.230835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:17.230835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:17.231431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:17.231583 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:17.232468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:17.232715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:17.232939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.233621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:54:17.233621 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.233621 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:17.233621 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.233621 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.233621 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:17.233621 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.233621 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.235153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.235566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:17.235911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.235911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.237890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:17.239014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.239014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:17.242775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:17.243347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:17.243558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:17.243780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:17.244594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:17.244843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.245577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:54:17.245577 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.245577 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:17.245577 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.245577 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.245577 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:17.245577 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.245577 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:17.247073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:17.247897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.247897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:17.247897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.247897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:17.247897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.247897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.247897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:17.247897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.247897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.248767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:17.249429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.250183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.250183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:17.250183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.250183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:17.250824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.251606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:17.252583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.252583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:17.252583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.252583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.252583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:17.252583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.252583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:17.278910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:17.279719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:17.279979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:17.280608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:17.281199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:17.281924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.283075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:54:17.283075 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.283075 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:17.283075 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.283075 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.283075 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:17.283075 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.283075 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:17.284956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:17.285499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.285499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:17.285499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.285499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:17.285499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.285499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.285499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:17.285499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.285499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:17.286093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.286093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:17.286093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:17.286093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:17.286093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.286093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:17.286093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:17.286093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:17.286093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:17.286093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.286093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:17.286527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.287051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.287051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:17.287051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.287051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.287443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:17.288003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.288721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.288721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:17.288721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.288721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:17.288875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:17.289550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:17.289698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:17.290618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:17.290618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:17.291002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:17.291294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.292141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:54:17.292141 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.292141 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:17.292141 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.292141 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.292141 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:17.292141 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.292141 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.293830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.294271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:17.294634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.294634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:17.296079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:17.296891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:17.297447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:17.297447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:17.297447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:22.186053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:22.187003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:22.187003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:22.188318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:22.226861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:22.227073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.227780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:22.227780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.227780 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:22.227780 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.227780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.227780 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:22.227780 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.227780 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.228757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.229225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:22.229660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:22.230177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.230800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:22.230800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.230800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.230800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:22.230800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.230800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:22.231472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:22.231768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:22.231866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:22.232537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:22.232867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:22.233121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.233826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:54:22.233826 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.233826 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:22.233826 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.233826 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.233826 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:22.233826 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.233826 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.234852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.235559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.236619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:22.237411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.238361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.238361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:22.238361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.238361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:22.243242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:22.244046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:22.244195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:22.244373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:22.244984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:22.245418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.246273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:54:22.246273 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.246273 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:22.246273 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.246273 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.246273 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:22.246273 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.246273 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:22.247276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:22.247737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.247737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:22.247737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.247737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:22.247737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.247737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.247737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:22.247737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.247737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.248182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:22.248579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.248941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.248941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:22.248941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.248941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:22.249283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.249909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:22.251006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.251006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:22.251006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.251006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.251006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:22.251006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.251006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:22.279544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:22.280133 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:22.280376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:22.280693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:22.281120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:22.281409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.281969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:54:22.281969 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.281969 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:22.281969 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.281969 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.281969 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:22.281969 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.281969 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:22.282799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:22.283585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.283585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:22.283585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.283585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:22.283585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.283585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.283585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:22.283585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.283585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:22.284417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.284417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:22.284417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:22.284417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:22.284417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.284417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:22.284417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:22.284417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:22.284417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:22.284417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.284417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:22.284773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:22.285244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:22.285662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.286140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:22.286140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:22.286140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.286140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:22.286140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.286140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:22.286140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.286140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.286140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:22.286140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.286140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:22.289272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:22.289648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:22.289840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:22.290099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:22.290614 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:22.291121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:22.291187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.291987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:54:22.291987 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.291987 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:22.291987 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.291987 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.291987 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:22.291987 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.291987 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.293336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.293942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:22.294534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:22.295491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:22.296460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:22.296460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:22.296460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:22.296460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:22.296460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:22.296460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:27.186879 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:27.187987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:27.187987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:27.189075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:27.225678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:27.226156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.227161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:27.227161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.227161 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:27.227161 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.227161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.227161 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:27.227161 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.227161 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.227952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.228689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:27.229494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:27.230438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.231096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:27.231096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.231096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.231096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:27.231096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.231096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:27.231971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:27.232678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:27.232822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:27.234001 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:27.234475 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:27.234730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.235435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:54:27.235435 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.235435 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:27.235435 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.235435 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.235435 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:27.235435 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.235435 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.236712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.237067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:27.237428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.237428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.238073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.238559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:27.239108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.239108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:27.243913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:27.244798 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:27.245006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:27.245221 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:27.245963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:27.246375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.247318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:54:27.247318 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.247318 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:27.247318 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.247318 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.247318 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:27.247318 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.247318 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:27.248893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:27.249727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.249727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:27.249727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.249727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:27.249727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.249727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.249727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:27.249727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.249727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.250549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:27.251179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:27.251874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.252630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.254254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:27.280142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:27.280747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:27.280865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:27.281020 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:27.281779 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:27.282301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.283166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:54:27.283166 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.283166 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:27.283166 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.283166 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.283166 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:27.283166 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.283166 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:27.284092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:27.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:27.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:27.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:27.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.284794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:27.285182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.285182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:27.285182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:27.285182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:27.285182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.285182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:27.285182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:27.285182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:27.285182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:27.285182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.285182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:27.285723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:27.286393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:27.286874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.287363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:27.287363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:27.287363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.287363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:27.287363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.287363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:27.287363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.287363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.287363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:27.287363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.287363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:27.289677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:27.290664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:27.290941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:27.291332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:27.291922 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:27.292341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:27.292587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.293513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:54:27.293513 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.293513 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:27.293513 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.293513 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.293513 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:27.293513 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.293513 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.295447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.295881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:27.296454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.296454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:27.296587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:27.297273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:27.298043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:27.298043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:27.298043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:32.187636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:32.188454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:32.188536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:32.189878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:32.229050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:32.229490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.230461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:54:32.230461 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.230461 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:32.230461 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.230461 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.230461 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:32.230461 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.230461 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.232203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.233025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:32.233747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:32.234654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.235572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:32.235572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.235572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.235572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:32.235572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.235572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:32.235572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:32.235572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:32.235572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:32.235572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:32.236698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:32.236698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.236698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:54:32.236698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.236698 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:32.236698 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.236698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.236698 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:32.236698 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.236698 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.237549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.237944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:32.238359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.238359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:32.238819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.239331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.240057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:32.240057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.240057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:32.244512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:32.245162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:32.245283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:32.245551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:32.246022 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:32.246278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.246911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:32.246911 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.246911 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:32.246911 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.246911 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.246911 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:32.246911 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.246911 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:32.248322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:32.248844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.248844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:32.248844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.248844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:32.248844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.248844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.248844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:32.248844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.248844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.249142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:32.249548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.250026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.250026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:32.250026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.250026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:32.250155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.250698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:32.251195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.251195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:32.251195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.251195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.251195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:32.251195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.251195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:32.280831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:32.281645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:32.281919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:32.282765 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:32.283083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:32.283506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.284493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:54:32.284493 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.284493 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:32.284493 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.284493 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.284493 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:32.284493 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.284493 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:32.286380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:32.287181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.287181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:32.287181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.287181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:32.287181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.287181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.287181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:32.287181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.287181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:32.287991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.287991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:32.287991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:32.287991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:32.287991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.287991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:32.287991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:32.287991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:32.287991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:32.287991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.287991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:32.288493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.289028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.289028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:32.289028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.289028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.289556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:32.290122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.290700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.290700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:32.290700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.290700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:32.290700 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:32.291043 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:32.291253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:32.291559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:32.292135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:32.292543 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:32.292786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.293861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:54:32.293861 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.293861 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:32.293861 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.293861 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.293861 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:32.293861 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.293861 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.295638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.296059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:32.296471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.296471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:32.296811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:32.297333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:32.297925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:32.297925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:32.297925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:37.188361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:37.189283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:37.189283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:37.189821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:37.228795 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:37.229341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.230515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:54:37.230515 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.230515 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:37.230515 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.230515 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.230515 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:37.230515 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.230515 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.231750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.232450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:37.233141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.233141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:37.233356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.234301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.235674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:37.235674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.235674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:37.235674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:37.237172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:37.237172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:37.237624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:37.237863 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:37.238031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.239406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:37.239406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.239406 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:37.239406 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.239406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.239406 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:37.239406 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.239406 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.239832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.240426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:37.241047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:37.241920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.242917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:37.242917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.242917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.242917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:37.242917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.242917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:37.245265 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:37.245895 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:37.246108 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:37.246387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:37.247320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:37.247646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.248472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:54:37.248472 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.248472 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:37.248472 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.248472 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.248472 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:37.248472 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.248472 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:37.250001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:37.250493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.250493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:37.250493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.250493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:37.250493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.250493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.250493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:37.250493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.250493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.251289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:37.251815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.252376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.252376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:37.252376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.252376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:37.252553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.253186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:37.253855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.253855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:37.253855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.253855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.253855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:37.253855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.253855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:37.281602 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:37.282810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:37.283077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:37.283677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:37.284242 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:37.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.285652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:54:37.285652 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.285652 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:37.285652 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.285652 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.285652 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:37.285652 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.285652 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:37.288863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:37.289397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.289397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:37.289397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.289397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:37.289397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.289397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.289397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:37.289397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.289397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:37.290142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.290142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:37.290142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:37.290142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:37.290142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.290142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:37.290142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:37.290142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:37.290142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:37.290142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.290142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:37.290520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.290934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.290934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:37.290934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.290934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.292268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:37.292790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.293359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.293359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:37.293359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.293359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:37.293359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:37.293359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:37.293359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:37.293359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:37.293919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:37.294156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:37.294356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.294971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:54:37.294971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.294971 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:37.294971 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.294971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.294971 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:37.294971 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.294971 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.297354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.298254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:37.298891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:37.299714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:37.301670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:37.301670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:37.301670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:37.301670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:37.301670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:37.301670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:42.189146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:42.189875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:42.189875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:42.191537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:42.194176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:54:42.227587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:42.228065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.228960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:54:42.228960 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.228960 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:42.228960 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.228960 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.228960 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:42.228960 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.228960 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.229752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.230420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:42.231088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:42.232136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.233742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:42.233742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.233742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.233742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:42.233742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.233742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:42.236498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:42.237150 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:42.237396 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:42.238386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:42.238721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:42.238959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.239746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:42.239746 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.239746 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:42.239746 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.239746 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.239746 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:42.239746 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.239746 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.240788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.241130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:42.241496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.241496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:42.241832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.242287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.242767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:42.242767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.242767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:42.245714 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:42.246372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:42.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:42.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:42.247520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:42.247845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.248690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:54:42.248690 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.248690 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:42.248690 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.248690 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.248690 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:42.248690 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.248690 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:42.249510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:42.250330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.251295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.251968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:42.252604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.253392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:42.254404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.254404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:42.254404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.254404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.254404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:42.254404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.254404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:42.282743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:42.284029 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:42.284029 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:42.284029 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:42.284522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:42.284901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.285821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:54:42.285821 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.285821 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:42.285821 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.285821 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.285821 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:42.285821 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.285821 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:42.287247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:42.287990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.287990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:42.287990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.287990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:42.287990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.287990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.287990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:42.287990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.287990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:42.288966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.288966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:42.288966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:42.288966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:42.288966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.288966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:42.288966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:42.288966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:42.288966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:42.288966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.288966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:42.289611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.290320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.290320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:42.290320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.290320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.291436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:42.292386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.294416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.294416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:42.294416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.294416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:42.294416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:42.294416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:42.294416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:42.294416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:42.294700 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:42.294700 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:42.294700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:42.297408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:42.297920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.298296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:42.298697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:42.299312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:42.299312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:42.299312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:42.299312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:42.299312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:42.299312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:42.299312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:42.299312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:42.299312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:42.299312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:46.461462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:46.462518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:46.462518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:46.472480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 21 13:54:46.473128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106267) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 21 13:54:46.473128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Fixing resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:54:46.504774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, resource={'id': 'b57c058f-054e-4a2a-980f-6bfda2111e90', 'name': 'private', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['6a3993e6-b5ba-4812-baa3-b25db5880cd9', 'c97deb27-a1d6-4e03-b489-95abde8fd791'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:29:32Z', 'updated_at': '2026-04-21T13:29:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 24380}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.505223 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.505817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3f6c78fa-b671-412d-bb34-5426a40d8bff, external_ids_update=None, columns={'tag': 24380}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.506003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.506895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:54:46.506895 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.506895 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:46.506895 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.506895 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.506895 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:46.506895 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.506895 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:46.508165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Failed to fix resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.508165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:46.508165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:54:46.508165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:54:46.508165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:54:46.508165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:54:46.508165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:54:46.508165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:46.508165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.508165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.508165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.508830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.509762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:46.509762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.509762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:46.509762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Fixing resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:54:46.542579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d0607710-681d-444d-be26-dbeca5a07b46, resource={'id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'name': 'public', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-21T13:29:38Z', 'updated_at': '2026-04-21T13:29:48Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.543015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d0607710-681d-444d-be26-dbeca5a07b46, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.543687 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-19f04b40-ed99-482a-8774-a76892a716d1, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.544024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.544966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:54:46.544966 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.544966 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:46.544966 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.544966 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.544966 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:46.544966 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.544966 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:46.546226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Failed to fix resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.546226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:46.546226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:54:46.546226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:54:46.546226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:54:46.546226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:54:46.546226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:54:46.546226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:46.546226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.546226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.546226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.546809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.547514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:46.547514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.547514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:46.547514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Fixing resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:54:46.576570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, resource={'id': '345e31fe-ede8-4dd0-823e-e81dcbc5bef0', 'name': 'admin_net', 'tenant_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'project_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['bb271c4b-4172-418f-995c-f72abd439dbe'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:34:46Z', 'updated_at': '2026-04-21T13:34:47Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 22486}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.576954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.577594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-c1b4e94e-49ea-4c1f-8edf-2f1433fdff16, external_ids_update=None, columns={'tag': 22486}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.577768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.578598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:54:46.578598 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.578598 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:46.578598 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.578598 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.578598 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:46.578598 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.578598 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:46.579808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Failed to fix resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.579808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:46.579808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:54:46.579808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:54:46.579808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:54:46.579808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:54:46.579808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:54:46.579808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:46.579808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.579808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.579808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.580493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.581136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:46.581136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.581136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:46.581136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Fixing resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:54:46.615448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, resource={'id': 'b364ad09-f70d-45d8-b9dd-8420f5f1c377', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['620e9559-2c15-4370-904f-f4677bc69542'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:35:24Z', 'updated_at': '2026-04-21T13:35:25Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 61006}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.615771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.616035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a100d4e8-7f11-40c9-8e4a-346c2c764f69, external_ids_update=None, columns={'tag': 61006}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.616136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.616850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:54:46.616850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.616850 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:46.616850 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.616850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.616850 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:46.616850 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.616850 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:46.617875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Failed to fix resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.617875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:46.617875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:54:46.617875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:54:46.617875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:54:46.617875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:54:46.617875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:54:46.617875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:46.617875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.617875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.617875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.618448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.619028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:46.619028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.619028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:46.619028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Fixing resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:54:46.692619 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, resource={'id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'enable_snat': True}, 'gw_port_id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-61m1f', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'ip_address': '172.24.5.49', 'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'}, {'ip_address': '2001:db8::162', 'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 3, 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04'}, resource_type=routers, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.692984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.693394 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.693723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.693931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.694843 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:54:46.694843 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.694843 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:46.694843 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.694843 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.694843 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:46.694843 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.694843 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:46.695289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Unable to update router 06899d4c-a436-45e4-bed5-95dabe888f51. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.696240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Failed to fix resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.696240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:46.696240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:54:46.696240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:54:46.696240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:54:46.696240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:54:46.696240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 21 13:54:46.696240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 21 13:54:46.696240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 21 13:54:46.696240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 21 13:54:46.696240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:46.696883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:46.698347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:46.698347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:46.698347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.698347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:46.698347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.698347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.698347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:46.698347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.698347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:46.698347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Fixing resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:54:46.798497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, resource={'id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'name': '', 'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'project_id': '', 'mac_address': 'fa:16:3e:fb:71:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000168688', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.798853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3 ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'neutron:network_name': 'neutron-d0607710-681d-444d-be26-dbeca5a07b46', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-06899d4c-a436-45e4-bed5-95dabe888f51'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.49/24', '2001:db8::162/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.799403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, lrouter_port=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:46.799760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.800862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Traceback (most recent call last):
Apr 21 13:54:46.800862 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.800862 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:46.800862 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.800862 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.800862 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:46.800862 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.800862 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:46.801989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] Maintenance task: Failed to fix resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.801989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:46.801989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:54:46.801989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:54:46.801989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:54:46.801989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:54:46.801989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 21 13:54:46.801989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:46.801989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.801989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.801989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:46.802407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:46.802815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:46.802815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:46.802815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:46.802815 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e44b81f6-be68-4a4d-a846-d8d64ab37b26 None None] OVN maintenance task check_for_inconsistencies finished in 0.340 seconds
Apr 21 13:54:47.189941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:47.190624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:47.190839 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:47.192127 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:47.226892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:47.227413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.228348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:54:47.228348 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.228348 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:47.228348 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.228348 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.228348 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:47.228348 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.228348 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.229108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.229717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:47.230418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:47.231229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.231833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:47.231833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.231833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.231833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:47.231833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.231833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:47.237103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:47.237627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:47.237762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:47.238423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:47.238633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:47.238803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.239975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:47.239975 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.239975 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:47.239975 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.239975 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.239975 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:47.239975 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.239975 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.241285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.241635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:47.241986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.241986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:47.243053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.243541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.244016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:47.244016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.244016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:47.246160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:47.246844 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:47.247053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:47.247312 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:47.247703 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:47.247904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.248533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:54:47.248533 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.248533 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:47.248533 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.248533 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.248533 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:47.248533 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.248533 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:47.249767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:47.250662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.250662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:47.250662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.250662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:47.250662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.250662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.250662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:47.250662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.250662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.251117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:47.251533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.251909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.251909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:47.251909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.251909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:47.252898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.253378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:47.253887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.253887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:47.253887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.253887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.253887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:47.253887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.253887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:47.283462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:47.284099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:47.284342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:47.284605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:47.285288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:47.285758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.286837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:54:47.286837 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.286837 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:47.286837 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.286837 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.286837 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:47.286837 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.286837 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:47.288346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:47.289258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.289258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:47.289258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.289258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:47.289258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.289258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.289258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:47.289258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.289258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:47.290162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.290162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:47.290162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:47.290162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:47.290162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.290162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:47.290162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:47.290162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:47.290162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:47.290162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.290162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:47.290793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.291421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.291421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:47.291421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.291421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.293742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:47.295173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.296448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.296448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:47.296448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.296448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:47.296448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:47.296448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:47.296448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:47.296448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:47.298355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:47.298355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:47.298355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.299366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:54:47.299366 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.299366 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:47.299366 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.299366 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.299366 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:47.299366 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.299366 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.300245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.300910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:47.301596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.301596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:47.301798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:47.302711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:47.303817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:47.303817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:47.303817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:52.190973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:52.192062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:52.192062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:52.193508 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:52.225975 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:52.226413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.227419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:54:52.227419 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.227419 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:52.227419 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.227419 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.227419 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:52.227419 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.227419 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.228411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.229159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:52.229964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:52.230998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.231932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:52.231932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.231932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.231932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:52.231932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.231932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:52.239531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:52.239531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:52.239531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:52.239531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:52.239834 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:52.240046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.240851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:52.240851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.240851 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:52.240851 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.240851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.240851 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:52.240851 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.240851 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.242477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.243183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:52.243914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.243914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:52.244019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.244760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.245598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:52.245598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.245598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:52.246823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:52.247609 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:52.247805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:52.248023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:52.248580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:52.248858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.249960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:54:52.249960 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.249960 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:52.249960 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.249960 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.249960 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:52.249960 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.249960 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:52.251673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:52.252168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.252168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:52.252168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.252168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:52.252168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.252168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.252168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:52.252168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.252168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.252506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:52.252909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.253299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.253299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:52.253299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.253299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:52.253689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.254167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:52.254738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.254738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:52.254738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.254738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.254738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:52.254738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.254738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:52.284145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:52.284822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:52.285084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:52.285382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:52.285999 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:52.286508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.287662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:54:52.287662 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.287662 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:52.287662 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.287662 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.287662 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:52.287662 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.287662 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:52.288464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:52.289101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.289101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:52.289101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.289101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:52.289101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.289101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.289101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:52.289101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.289101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:52.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:52.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:52.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:52.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:52.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:52.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:52.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:52.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:52.290011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:52.291137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:52.292155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.293003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:52.293003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:52.293003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.293003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:52.293003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.293003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:52.293003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.293003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.293003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:52.293003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.293003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:52.294257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:52.295088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:52.295088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:52.295088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:52.295531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:52.295926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:52.296226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:52.298333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:52.299071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.299501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:52.300001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:52.300671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:52.300671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:52.300671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:52.300671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:52.300671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:52.300671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:52.300671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:52.300671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:52.300671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:52.300671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:57.191545 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:57.192676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:57.192676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:57.193466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:57.231170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:57.231352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.232082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:54:57.232082 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.232082 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:57.232082 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.232082 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.232082 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:57.232082 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.232082 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.233603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.234350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:57.234943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:57.235684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.236509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:57.236509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.236509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.236509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:57.236509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.236509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:57.238463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:57.239167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:57.239491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:57.240697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:57.240930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:57.241386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.241813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:54:57.241813 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.241813 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:57.241813 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.241813 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.241813 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:57.241813 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.241813 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.243538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.243887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:57.244289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.244289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:57.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.245899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.246480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:57.246480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.246480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:57.247491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:57.247978 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:57.248281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:57.248618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:54:57.249158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:57.249595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.250613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:54:57.250613 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.250613 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:57.250613 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.250613 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.250613 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:57.250613 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.250613 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:57.251989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:57.252512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.252512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:57.252512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.252512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:57.252512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.252512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.252512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:57.252512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.252512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.253980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:57.254448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.254870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.254870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:57.254870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.254870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:57.256405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.256891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:57.257419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.257419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:57.257419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.257419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.257419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:57.257419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.257419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:57.284674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:57.285346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:57.285613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:57.286015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:57.286704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:57.287198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.288006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:54:57.288006 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.288006 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:57.288006 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.288006 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.288006 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:57.288006 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.288006 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:54:57.289573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:54:57.290342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.290342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:54:57.290342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.290342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:54:57.290342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.290342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.290342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:54:57.290342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.290342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:54:57.291246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.291246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:57.291246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:57.291246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:57.291246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.291246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:57.291246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:57.291246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:57.291246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:57.291246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.291246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:57.291626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.292080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.292080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:57.292080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.292080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.292406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:57.292901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.293588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.293588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:57.293588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.293588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:54:57.294779 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:54:57.295587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:54:57.295901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:54:57.296402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:54:57.296831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:57.297178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:54:57.297461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.298446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:54:57.298446 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.298446 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:54:57.298446 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.298446 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.298446 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:54:57.298446 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.298446 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.299522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.299913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:54:57.300326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.300326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:54:57.300647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:54:57.301085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:54:57.301618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:54:57.301618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:54:57.301618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:02.192246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:02.193883 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:02.193883 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:02.194889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:02.234693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:02.235068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.236055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:02.236055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.236055 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:02.236055 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.236055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.236055 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:02.236055 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.236055 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.236867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.237530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:02.238192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:02.238969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.239904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:02.239904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.239904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.239904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:02.239904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.239904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:02.239904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:02.239904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:02.239904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:02.240850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:02.241249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:02.241492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.242436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:02.242436 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.242436 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:02.242436 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.242436 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.242436 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:02.242436 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.242436 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.244056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.244402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:02.244727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.244727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:02.246037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.246530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.247071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:02.247071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.247071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:02.247926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:02.249009 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:02.249009 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:02.249009 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:02.249346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:02.249523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.250162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:55:02.250162 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.250162 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:02.250162 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.250162 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.250162 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:02.250162 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.250162 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:02.252089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:02.252552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.252552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:02.252552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.252552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:02.252552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.252552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.252552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:02.252552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.252552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.253496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:02.253856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.254193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.254193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:02.254193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.254193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:02.255295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.255710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:02.256173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.256173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:02.256173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.256173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.256173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:02.256173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.256173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:02.285419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:02.285971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:02.286096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:02.286202 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:02.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:02.287564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.288579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:55:02.288579 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.288579 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:02.288579 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.288579 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.288579 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:02.288579 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.288579 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:02.289519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:02.290452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:02.291373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:02.292249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:02.294164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.295542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:02.296314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:02.296314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:02.296314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:02.296660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:02.297092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:02.297411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:02.297564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.298108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:55:02.298108 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.298108 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:02.298108 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.298108 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.298108 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:02.298108 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.298108 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.298947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.299417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:02.299869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:02.300448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:02.301134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:02.301134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:02.301134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:02.301134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:02.301134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:02.301134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:07.192905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:07.194253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:07.194253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:07.194747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:07.229873 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:07.230328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.231234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:07.231234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.231234 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:07.231234 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.231234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.231234 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:07.231234 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.231234 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.232299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.233055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:07.233940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:07.234940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.235894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:07.235894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.235894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.235894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:07.235894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.235894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:07.239424 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:07.240067 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:07.240321 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:07.241483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:07.241824 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:07.243141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.243141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:07.243141 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.243141 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:07.243141 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.243141 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.243141 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:07.243141 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.243141 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.244321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.244864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:07.245234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.245234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:07.246095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.246596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.247061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:07.247061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.247061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:07.248880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:07.249745 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:07.250121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:07.250434 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:07.250884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:07.251079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.251773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:55:07.251773 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.251773 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:07.251773 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.251773 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.251773 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:07.251773 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.251773 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:07.253237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:07.253743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.253743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:07.253743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.253743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:07.253743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.253743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.253743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:07.253743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.253743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.254391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:07.254791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.255189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.255189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:07.255189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.255189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:07.255611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.256077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:07.256627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.256627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:07.256627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.256627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.256627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:07.256627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.256627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:07.286099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:07.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:07.286878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:07.287201 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:07.287915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:07.288301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.289647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:55:07.289647 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.289647 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:07.289647 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.289647 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.289647 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:07.289647 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.289647 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:07.291073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:07.291566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.291566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:07.291566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.291566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:07.291566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.291566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.291566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:07.291566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.291566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:07.292869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.292869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:07.292869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:07.292869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:07.292869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.292869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:07.292869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:07.292869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:07.292869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:07.292869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.292869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:07.293314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.293724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.293724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:07.293724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.293724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.295283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:07.295787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.296329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.296329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:07.296329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.296329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:07.296329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:07.296329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:07.296329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:07.296613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:07.296935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:07.297199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:07.297394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.298004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:55:07.298004 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.298004 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:07.298004 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.298004 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.298004 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:07.298004 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.298004 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.299434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.299832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:07.300196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.300196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:07.301011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:07.301714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:07.302461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:07.302461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:07.302461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:12.193532 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:12.194342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:12.194342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:12.195530 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:12.234243 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:12.234637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.235661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:12.235661 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.235661 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:12.235661 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.235661 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.235661 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:12.235661 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.235661 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.237138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.237799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:12.238541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.238541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:12.238746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.239690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.240650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:12.240650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.240650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:12.240650 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:12.240650 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:12.240930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:12.241753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:12.242129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:12.242373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.243154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:12.243154 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.243154 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:12.243154 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.243154 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.243154 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:12.243154 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.243154 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.244742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.245125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:12.245501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.245501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:12.246845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.247390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.247924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:12.247924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.247924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:12.249470 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:12.250027 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:12.250178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:12.250368 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:12.250875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:12.251167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.251873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:55:12.251873 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.251873 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:12.251873 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.251873 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.251873 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:12.251873 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.251873 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:12.252849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:12.253557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.253557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:12.253557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.253557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:12.253557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.253557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.253557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:12.253557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.253557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.253963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:12.254433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.254831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.254831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:12.254831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.254831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:12.255085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.255605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:12.256132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.256132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:12.256132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.256132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.256132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:12.256132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.256132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:12.287972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:12.288617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:12.288818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:12.289031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:12.289630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:12.289963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.290870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:55:12.290870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.290870 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:12.290870 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.290870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.290870 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:12.290870 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.290870 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:12.292462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:12.293055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.293055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:12.293055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.293055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:12.293055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.293055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.293055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:12.293055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.293055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:12.294093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.294093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:12.294093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:12.294093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:12.294093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.294093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:12.294093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:12.294093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:12.294093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:12.294093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.294093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:12.294647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.295165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.295165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:12.295165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.295165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.295736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:12.296299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.296949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.296949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:12.296949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.296949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:12.296949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:12.296949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:12.296949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:12.297422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:12.297658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:12.298013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:12.298300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.298995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:55:12.298995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.298995 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:12.298995 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.298995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.298995 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:12.298995 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.298995 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.300011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.300902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:12.301986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:12.302607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:12.302607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:12.302607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:12.302607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:12.302607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:12.302607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:17.194382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:17.194963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:17.195181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:17.196460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:17.231095 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:17.231478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.232023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:17.232023 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.232023 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:17.232023 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.232023 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.232023 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:17.232023 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.232023 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.232818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.233499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:17.234104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:17.234832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.235718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:17.235718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.235718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.235718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:17.235718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.235718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:17.240670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:17.241283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:17.241417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:17.242150 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:17.242473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:17.242683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.243331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:17.243331 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.243331 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:17.243331 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.243331 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.243331 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:17.243331 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.243331 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.244410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.244796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:17.245141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.245141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:17.245495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.245931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.246524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:17.246524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.246524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:17.249958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:17.250637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:17.250849 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:17.251070 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:17.251501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:17.251698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.252322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:55:17.252322 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.252322 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:17.252322 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.252322 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.252322 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:17.252322 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.252322 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:17.253704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:17.254551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.254551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:17.254551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.254551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:17.254551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.254551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.254551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:17.254551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.254551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.255254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:17.255620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.255999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.255999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:17.255999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.255999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:17.256394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.256829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:17.257297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.257297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:17.257297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.257297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.257297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:17.257297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.257297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:17.288487 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:17.289093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:17.289598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:17.289970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:17.290945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:17.291300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.292091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:55:17.292091 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.292091 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:17.292091 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.292091 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.292091 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:17.292091 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.292091 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:17.293557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:17.294550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.294550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:17.294550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.294550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:17.294550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.294550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.294550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:17.294550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.294550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:17.295475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.295475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:17.295475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:17.295475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:17.295475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.295475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:17.295475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:17.295475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:17.295475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:17.295475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.295475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:17.295896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.296270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.296270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:17.296270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.296270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.296603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:17.297122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.297648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.297648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:17.297648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.297648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:17.297648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:17.297648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:17.297648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:17.297884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:17.298404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:17.298735 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:17.298953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.299875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:55:17.299875 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.299875 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:17.299875 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.299875 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.299875 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:17.299875 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.299875 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.301450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.301801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:17.302189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.302189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:17.303190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:17.303676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:17.304270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:17.304270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:17.304270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:22.196687 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:22.197323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:22.197646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:22.198880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:22.240920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:22.241269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.242437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:22.242437 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.242437 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:22.242437 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.242437 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.242437 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:22.242437 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.242437 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:22.243149 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.245674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.246520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:22.247104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.247104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:22.247663 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:22.248837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.249720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.250637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:22.250637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.250637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:22.250637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:22.250853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:22.251239 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:22.252178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:22.252451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:22.252882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:22.253105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.253942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:22.254498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:22.254498 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.254498 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:22.254498 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.254498 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.254498 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:22.254498 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.254498 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:22.255122 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.258687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.259460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:22.260120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.260120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:22.260120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:22.261423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.261947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.262507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:22.262507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.262507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:22.262507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:55:22.262507 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.262507 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:22.262507 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.262507 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.262507 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:22.262507 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.262507 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:22.262817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:22.263407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.263407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:22.263407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.263407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:22.263407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.263407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.263407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:22.263407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.263407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.263796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:22.264261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.264679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.264679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:22.264679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.264679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:22.264910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.265365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:22.265972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.265972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:22.265972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.265972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.265972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:22.265972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.265972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:22.289124 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:22.289720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:22.289990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:22.290311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:22.291075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:22.291627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.292710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:55:22.292710 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.292710 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:22.292710 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.292710 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.292710 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:22.292710 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.292710 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:22.294354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:22.295161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.295161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:22.295161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.295161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:22.295161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.295161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.295161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:22.295161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.295161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:22.296126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.296126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:22.296126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:22.296126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:22.296126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.296126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:22.296126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:22.296126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:22.296126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:22.296126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.296126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:22.296583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.296964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.296964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:22.296964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.296964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.298433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:22.299534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.300507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.300507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:22.300507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.300507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:22.300507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:22.300507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:22.300507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:22.300507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:22.300965 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:22.300965 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:22.300965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.300965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:55:22.300965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.300965 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:22.300965 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.300965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.300965 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:22.300965 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.300965 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.301971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.302475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:22.302859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.302859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:22.302963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:22.303463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:22.303965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:22.303965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:22.303965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:27.197419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:27.198904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:27.198904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:27.199397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:27.236488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:27.237102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.238295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:27.238295 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.238295 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:27.238295 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.238295 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.238295 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:27.238295 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.238295 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.239978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.241029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:27.242066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:27.243445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.244302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:27.244302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.244302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.244302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:27.244302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.244302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:27.244720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:27.245379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:27.245624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:27.246546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:27.246980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:27.247077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.247839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:27.247839 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.247839 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:27.247839 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.247839 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.247839 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:27.247839 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.247839 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.249472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.249822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:27.250163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.250163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:27.251343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.251795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.252308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:27.252308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.252308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:27.252308 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:27.252308 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:27.252308 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:27.252528 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:27.252972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:27.253297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.254241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:55:27.254241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.254241 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:27.254241 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.254241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.254241 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:27.254241 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.254241 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:27.255375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:27.256698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.257428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.258067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:27.258712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.259858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:27.260902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.260902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:27.260902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.260902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.260902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:27.260902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.260902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:27.289841 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:27.291072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:27.291072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:27.291340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:27.292067 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:27.292583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.293599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:55:27.293599 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.293599 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:27.293599 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.293599 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.293599 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:27.293599 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.293599 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:27.295454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:27.296331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.296331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:27.296331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.296331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:27.296331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.296331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.296331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:27.296331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.296331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:27.297185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.297185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:27.297185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:27.297185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:27.297185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.297185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:27.297185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:27.297185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:27.297185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:27.297185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.297185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:27.297889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.298522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.298522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:27.298522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.298522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.299630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:27.300149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.300679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.300679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:27.300679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.300679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:27.300679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:27.301190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:27.301349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:27.301575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:27.302374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:27.302782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:27.303025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.304000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:55:27.304000 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.304000 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:27.304000 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.304000 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.304000 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:27.304000 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.304000 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.305928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.306324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:27.306714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.306714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:27.310896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:27.311473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:27.311969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:27.311969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:27.311969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:32.227853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:32.228686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:32.228900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:32.229931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:32.245412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:32.246430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:32.246430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:32.248704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:32.249263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:32.250825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.252145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:32.252145 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.252145 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:32.252145 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.252145 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.252145 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:32.252145 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.252145 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.256493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.257264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:32.257775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.257775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:32.257775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:32.260391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.261306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.262087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:32.262087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.262087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:32.262087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:32.262087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:32.262087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:32.263814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:32.264291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.266322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:55:32.266322 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.266322 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:32.266322 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.266322 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.266322 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:32.266322 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.266322 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:32.268596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:32.269136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.269136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:32.269136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.269136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:32.269136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.269136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.269136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:32.269136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.269136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.271128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:32.271714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.272353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.272353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:32.272353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.272353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:32.274137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.274883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:32.275728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.275728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:32.275728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.275728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.275728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:32.275728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.275728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:32.284911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:32.285353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.286229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:32.286229 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.286229 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:32.286229 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.286229 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.286229 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:32.286229 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.286229 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.287371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.287752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:32.288168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.288168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:32.288570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.290516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:32.290516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.290516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:32.290516 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:32.291101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:32.291331 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:32.291624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:32.292347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:32.292873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.293762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:55:32.293762 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.293762 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:32.293762 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.293762 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.293762 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:32.293762 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.293762 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:32.295327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:32.296335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.296335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:32.296335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.296335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:32.296335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.296335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.296335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:32.296335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.296335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:32.296958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.296958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:32.296958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:32.296958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:32.296958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.296958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:32.296958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:32.296958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:32.296958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:32.296958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.296958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:32.297561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.297948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.297948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:32.297948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.297948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.298068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:32.298574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.299141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.299141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:32.299141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.299141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:32.301195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:32.301474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:32.301591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:32.301777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:32.302080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:32.302346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:32.302538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.303012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:55:32.303012 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.303012 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:32.303012 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.303012 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.303012 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:32.303012 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.303012 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.303903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.304305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:32.304718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.304718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:32.305133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:32.305634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:32.306239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:32.306239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:32.306239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:37.228723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:37.229509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:37.229609 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:37.230891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:37.264302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:37.264302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:37.264302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:37.264302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:37.264302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:37.264302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:37.264743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:37.265758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.268903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.270760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:37.271480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.272005 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:37.272429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:37.272901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.273360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.273629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:37.273931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.274311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:37.274680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.274680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:37.274680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.274680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.274680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:37.274680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.274680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:37.285114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:37.285114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.285349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:37.285349 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.285349 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:37.285349 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.285349 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.285349 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:37.285349 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.285349 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.056 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.286200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.287195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:37.287897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:37.288805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.289961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:37.289961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.289961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.289961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:37.289961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.289961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:37.291157 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:37.291823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:37.292010 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:37.292310 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:37.292712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:37.292932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.293531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:55:37.293531 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.293531 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:37.293531 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.293531 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.293531 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:37.293531 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.293531 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:37.294860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:37.295159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.295159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:37.295159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.295159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:37.295159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.295159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.295159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:37.295159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.295159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:37.296690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.296690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:37.296690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:37.296690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:37.296690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.296690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:37.296690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:37.296690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:37.296690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:37.296690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.296690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:37.296973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.297231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.297231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:37.297231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.297231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.297804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:37.298194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.298687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.298687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:37.298687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.298687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:37.301696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:37.302177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:37.302417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:37.302733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:37.303344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:37.303542 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:37.303694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.305750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:55:37.305750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.305750 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:37.305750 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.305750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.305750 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:37.305750 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.305750 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.307038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.307909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:37.308577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:37.309327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:37.310219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:37.310219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:37.310219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:37.310219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:37.310219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:37.310219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:42.229541 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:42.230112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:42.230284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:42.231370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:42.234490 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=434 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:55:42.246339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:42.247009 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:42.247317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:42.249393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:42.249964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:42.250100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.254438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:42.254438 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.254438 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:42.254438 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.254438 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.254438 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:42.254438 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.254438 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.256713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.257496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:42.258169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:42.259001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.260079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:42.260079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.260079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.260079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:42.260079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.260079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:42.260079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:42.261053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:42.262702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:42.262962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:42.263904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:42.265187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.267293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:42.267293 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.267293 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:42.267293 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.267293 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.267293 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:42.267293 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.267293 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:42.269028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:42.269539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.269539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:42.269539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.269539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:42.269539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.269539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.269539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:42.269539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.269539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.270551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:42.271127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.271737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.271737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:42.271737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.271737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:42.274791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.275501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:42.276345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.276345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:42.276345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.276345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.276345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:42.276345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.276345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.060 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:42.302362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.302720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.303302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:42.303778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:42.304744 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:42.305156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:42.305156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:42.305156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:42.305156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:42.305511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:42.306516 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:42.307171 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:42.307538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.308548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:42.308862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:55:42.308862 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.308862 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:42.308862 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.308862 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.308862 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:42.308862 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.308862 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:42.309987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:42.310350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.310350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:42.310350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.310350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:42.310350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.310350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.310350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:42.310350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.310350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:42.310350 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:42.311474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.311474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:42.311474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:42.311474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:42.311474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.311474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:42.311474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:42.311474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:42.311474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:42.311474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.311474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:42.311717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.311985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.311985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:42.311985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.311985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:42.311985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.312688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:42.313022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.313376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.313376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:42.313376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.313376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:42.313376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.313472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:55:42.313472 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.313472 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:42.313472 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.313472 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.313472 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:42.313472 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.313472 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_mac_aging_settings failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.314564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.314834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:42.315094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.315094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:42.315945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:42.316270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:42.316611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:42.316611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:42.316611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:47.230357 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:47.231151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:47.231595 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:47.232915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:47.249523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:47.251147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:47.253336 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:47.254700 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:47.255283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:47.255595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.257924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:47.257924 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.257924 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:47.257924 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.257924 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.257924 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:47.257924 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.257924 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.260398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.261039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:47.261762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.261762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:47.262522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.263370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.264236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:47.264236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.264236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:47.264236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:47.264236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:47.264236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:47.264236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:47.265524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:47.265846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.267955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:47.267955 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.267955 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:47.267955 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.267955 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.267955 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:47.267955 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.267955 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:47.268829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:47.269260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.269260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:47.269260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.269260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:47.269260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.269260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.269260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:47.269260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.269260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.270605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:47.270971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.271416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.271416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:47.271416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.271416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:47.272920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.273392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:47.274101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.274101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:47.274101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.274101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.274101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:47.274101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.274101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:47.282399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:47.282763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.283743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:47.283743 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.283743 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:47.283743 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.283743 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.283743 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:47.283743 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.283743 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.284778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.285358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:47.285949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:47.286610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.287456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:47.287456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.287456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.287456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:47.287456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.287456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:47.304347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:47.304797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:47.305017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:47.305234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:47.305937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:47.306145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:47.306581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.307131 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:47.308125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:47.308125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:47.308587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:55:47.308587 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.308587 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:47.308587 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.308587 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.308587 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:47.308587 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.308587 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:47.310090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:47.310992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.310992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:47.310992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.310992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:47.310992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.310992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.310992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:47.310992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.310992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:47.310992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:47.312796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.312796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:47.312796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:47.312796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:47.312796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.312796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:47.312796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:47.312796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:47.312796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:47.312796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.312796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:47.313383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.314076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.314076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:47.314076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.314076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:47.314076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.316457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:47.316936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.317662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.317662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:47.317662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.317662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:47.317990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.318625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:55:47.318625 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.318625 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:47.318625 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.318625 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.318625 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:47.318625 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.318625 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.321946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:47.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.322790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:47.323967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:47.324696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:47.325231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:47.325231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:52.231070 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:52.232279 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:52.232279 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:52.233561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:52.251743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:52.252803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:52.253034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:52.255911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:52.259757 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:52.259757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.259983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:52.259983 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.259983 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:52.259983 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.259983 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.259983 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:52.259983 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.259983 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.264027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.264526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:52.264972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.264972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:52.266191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.266801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.267446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:52.267446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.267446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:52.267446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:52.267661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:52.267765 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:52.267902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:52.269960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:52.270204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.271099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:52.271099 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.271099 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:52.271099 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.271099 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.271099 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:52.271099 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.271099 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:52.273243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:52.273962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.273962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:52.273962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.273962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:52.273962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.273962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.273962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:52.273962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.273962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.277494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:52.278151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.278786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.278786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:52.278786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.278786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:52.279902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.280771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:52.281739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.281739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:52.281739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.281739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.281739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:52.281739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.281739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:52.283848 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:52.284105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.284792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:52.284792 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.284792 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:52.284792 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.284792 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.284792 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:52.284792 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.284792 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.285945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.286340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:52.286701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.286701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:52.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.287724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.288363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:52.288363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.288363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:52.304910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:52.305744 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:52.305940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:52.306176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:52.307120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:52.307907 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:52.308149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.309380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:52.309965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:55:52.309965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.309965 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:52.309965 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.309965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.309965 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:52.309965 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.309965 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:52.310552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:52.312478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:52.313276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.313276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:52.313276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.313276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:52.313276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.313276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.313276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:52.313276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.313276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:52.313276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:52.313981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.313981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:52.313981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:52.313981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:52.313981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.313981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:52.313981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:52.313981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:52.313981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:52.313981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.313981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:52.314600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.315449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.315449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:52.315449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.315449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.315931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:52.316574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.317147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.317147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:52.317147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.317147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:52.317147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:52.317147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:52.317147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.317706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:55:52.317706 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.317706 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:52.317706 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.317706 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.317706 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:52.317706 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.317706 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.319155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.319616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:52.320006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.320006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:52.320274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:52.320731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:52.321310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:52.321310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:52.321310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:57.232117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:57.233058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:57.233058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:57.234176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:57.252675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:57.254386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:57.254736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:57.257591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:57.257952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:57.258244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.260708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:55:57.260708 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.260708 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:57.260708 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.260708 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.260708 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:57.260708 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.260708 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.262049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.266184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:57.267934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:57.269939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.271019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:57.271019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.271019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.271019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:57.271019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.271019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:57.271019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:57.271019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:57.271019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:57.271019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:55:57.271598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:57.271598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.272136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:55:57.272136 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.272136 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:57.272136 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.272136 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.272136 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:57.272136 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.272136 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:57.274479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:57.276299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.276299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:57.276299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.276299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:57.276299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.276299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.276299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:57.276299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.276299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.276764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:57.277111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.277490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.277490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:57.277490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.277490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:57.280143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.281073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:57.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:57.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:57.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:57.286234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:57.286574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.287441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:55:57.287441 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.287441 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:57.287441 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.287441 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.287441 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:57.287441 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.287441 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.288516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.288887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:57.289270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.289270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:57.289603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.290145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.290681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:57.290681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.290681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:57.305642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:57.306118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:57.306387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:57.306618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:57.307161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:57.307711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.308556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:55:57.308556 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.308556 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:57.308556 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.308556 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.308556 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:57.308556 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.308556 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:55:57.309755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:55:57.310181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.310181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:55:57.310181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.310181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:55:57.310181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.310181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.310181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:55:57.310181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.310181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:55:57.310658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.310658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:57.310658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:57.310658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:57.310658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.310658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:57.310658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:57.310658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:57.310658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:57.310658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.310658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:57.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.311360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.311360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:57.311360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.311360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.311964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:57.312439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.312912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.312912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:57.312912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.312912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:55:57.312912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:55:57.312912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:55:57.313166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:55:57.313477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:55:57.313945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:57.314550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:55:57.314793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.316850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:55:57.316850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.316850 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:55:57.316850 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.316850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.316850 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:55:57.316850 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.316850 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.319283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.319754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:55:57.320110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.320110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:55:57.321399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:55:57.321956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:55:57.322499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:55:57.322499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:55:57.322499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:02.233129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:02.234296 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:02.234296 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:02.235592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:02.254803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:02.257508 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:02.257969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:02.261862 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:02.263403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:02.263593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.264528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:56:02.264528 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.264528 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:02.264528 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.264528 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.264528 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:02.264528 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.264528 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.267514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.268012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:02.268433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.268433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:02.270884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.272275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.273686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:02.273686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.273686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:02.273686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:02.273686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:02.273686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:02.273686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:02.277245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:02.277730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.278637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:56:02.278637 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.278637 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:02.278637 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.278637 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.278637 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:02.278637 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.278637 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:02.279505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:02.279991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.279991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:02.279991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.279991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:02.279991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.279991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.279991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:02.279991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.279991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.282126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:02.282622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.283047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.283047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:02.283047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.283047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:02.284191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.284661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:02.285183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.285183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:02.285183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.285183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.285183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:02.285183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.285183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:02.287048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:02.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.288366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:02.288366 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.288366 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:02.288366 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.288366 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.288366 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:02.288366 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.288366 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.289546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.290081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:02.290570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.290570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:02.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.291443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.292192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:02.292192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.292192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:02.306575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:02.307435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:02.307435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:02.307620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:02.308320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:02.308574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.309589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:56:02.309589 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.309589 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:02.309589 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.309589 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.309589 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:02.309589 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.309589 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:02.311443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:02.311858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.311858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:02.311858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.311858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:02.311858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.311858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.311858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:02.311858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.311858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:02.313247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.313754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.313754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:02.313754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.313754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.314978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:02.315497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.316079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.316079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:02.316079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.316079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:02.316079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:02.316079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:02.316340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:02.316648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:02.317302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:02.317698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:02.317952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.318807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:56:02.318807 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.318807 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:02.318807 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.318807 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.318807 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:02.318807 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.318807 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.319993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.320358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:02.320713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.320713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:02.321149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:02.321623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:02.322146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:02.322146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:02.322146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:07.233666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:07.236743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:07.236743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:07.236743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:07.257689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:07.258306 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:07.258855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:07.261173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:07.262568 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:07.263058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.264253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:07.264253 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.264253 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:07.264253 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.264253 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.264253 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:07.264253 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.264253 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.268131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.268569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:07.268907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.268907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:07.272888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.273420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.273914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:07.273914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.273914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:07.275296 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:07.276931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:07.277323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:07.279080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:07.279646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:07.279914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.283574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:07.283574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.283574 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:07.283574 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.283574 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.283574 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:07.283574 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.283574 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:07.286390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:07.287134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.287134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:07.287134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.287134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:07.287134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.287134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.287134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:07.287134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.287134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.289278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:07.289747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.290684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.290684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:07.290684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.290684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:07.291226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.291913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:07.292417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.292417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:07.292417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.292417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.292417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:07.292417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.292417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:07.293337 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:07.293337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.293677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:56:07.293677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.293677 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:07.293677 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.293677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.293677 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:07.293677 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.293677 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.059 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.294607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.295171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:07.295769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:07.296548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.297068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:07.297068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.297068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.297068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:07.297068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.297068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:07.307049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:07.307569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:07.307859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:07.308073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:07.308674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:07.308997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.309905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:56:07.309905 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.309905 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:07.309905 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.309905 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.309905 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:07.309905 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.309905 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:07.311047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:07.312074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.312074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:07.312074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.312074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:07.312074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.312074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.312074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:07.312074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.312074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:07.312702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.312702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:07.312702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:07.312702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:07.312702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.312702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:07.312702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:07.312702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:07.312702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:07.312702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.312702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:07.314047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:07.315559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:07.316791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:07.320473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:07.321366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:07.321366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:07.321366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:07.321366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:07.321366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.321366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 13:56:07.321366 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.321366 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:07.321366 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:07.321737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.322741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.323797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:07.323797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.323797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:07.325056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:07.325639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:07.326094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:07.326094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:07.326094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:12.234956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:12.235967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:12.235967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:12.237065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:12.259044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:12.260711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:12.263490 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:12.266000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:12.267591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:12.267763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.269606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:12.269606 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.269606 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:12.269606 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.269606 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.269606 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:12.269606 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.269606 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.272417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.272868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:12.273328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.273328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:12.276194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.276783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.277424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:12.277424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.277424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:12.277424 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:12.278236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:12.278389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:12.278533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:12.279237 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:12.280411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.281276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:56:12.281276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.281276 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:12.281276 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.281276 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.281276 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:12.281276 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.281276 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:12.284206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:12.284740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.284740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:12.284740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.284740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:12.284740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.284740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.284740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:12.284740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.284740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.286842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:12.287356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.287928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.287928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:12.287928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.287928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:12.288961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.289511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:12.290133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.290133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:12.290133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.290133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.290133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:12.290133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.290133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:12.290133 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:12.290133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.290421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:12.290421 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.290421 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:12.290421 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.290421 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.290421 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:12.290421 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.290421 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.291813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.292184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:12.292549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.292549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:12.293647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.294113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.294776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:12.294776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.294776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:12.307555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:12.308136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:12.308383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:12.308612 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:12.309165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:12.309578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.310394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:56:12.310394 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.310394 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:12.310394 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.310394 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.310394 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:12.310394 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.310394 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:12.312304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:12.313095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.313095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:12.313095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.313095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:12.313095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.313095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.313095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:12.313095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.313095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:12.313729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.313729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:12.313729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:12.313729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:12.313729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.313729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:12.313729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:12.313729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:12.313729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:12.313729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.313729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:12.314344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.315138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.315138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:12.315138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.315138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.315673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:12.316607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.317141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.317141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:12.317141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.317141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:12.317934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:12.318421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:12.318660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:12.319005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:12.319538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:12.319876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:12.320089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.320959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 13:56:12.320959 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.320959 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:12.320959 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.320959 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.320959 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:12.320959 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.320959 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.322370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:12.323283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.323283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:12.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:12.324579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:12.325107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:12.325107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:12.325107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:17.235988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:17.236861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:17.236861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:17.237949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:17.260730 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:17.262550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:17.262871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:17.264707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:17.265127 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:17.265341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.267610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:56:17.267610 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.267610 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:17.267610 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.267610 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.267610 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:17.267610 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.267610 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.269482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.270121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:17.271310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.271310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:17.274041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.274705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.275294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:17.275294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.275294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:17.278147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:17.278832 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:17.279459 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:17.279672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:17.280384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:17.280918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.282545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:17.282545 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.282545 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:17.282545 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.282545 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.282545 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:17.282545 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.282545 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.283580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.283936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:17.284347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.284347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:17.284347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:17.285862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.286366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.286840 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:17.287788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:17.288271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.288271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:17.288271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.288271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:17.288271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.288271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.288271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:17.288271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.288271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.288544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:17.290198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:17.290600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.290954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.292510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:17.308020 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:17.308772 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:17.309075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:17.309423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:17.309919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:17.310233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.311167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:56:17.311167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.311167 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:17.311167 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.311167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.311167 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:17.311167 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.311167 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:17.312717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:17.313395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.313395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:17.313395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.313395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:17.313395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.313395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.313395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:17.313395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.313395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:17.314149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.314149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:17.314149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:17.314149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:17.314149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.314149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:17.314149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:17.314149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:17.314149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:17.314149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.314149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:17.315066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.315862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.315862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:17.315862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.315862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.320269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:17.321035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.322664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.322664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:17.322664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.322664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:17.322664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:17.322664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:17.322664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:17.322664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:17.323155 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:17.323155 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:17.323155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.323155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:56:17.323155 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.323155 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:17.323155 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.323155 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.323155 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:17.323155 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.323155 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.323840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.324525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:17.325098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.325098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:17.325807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:17.326637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:17.327784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:17.327784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:17.327784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:22.236753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:22.237905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:22.237905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:22.238869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:22.263015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:22.263559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:22.263817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:22.267760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:22.268091 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:22.269449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.271747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:22.271747 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.271747 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:22.271747 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.271747 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.271747 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:22.271747 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.271747 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.275035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.275850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:22.276595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.276595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:22.278578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.279509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.280532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:22.280532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.280532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:22.280532 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:22.281808 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:22.281908 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:22.282060 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:22.282554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:22.282757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.283698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:22.283698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.283698 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:22.283698 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.283698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.283698 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:22.283698 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.283698 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:22.284572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:22.285079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.285079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:22.285079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.285079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:22.285079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.285079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.285079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:22.285079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.285079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.287054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:22.287451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.287839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.287839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:22.287839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.287839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:22.288319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.288761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:22.289322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.289322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:22.289322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.289322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.289322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:22.289322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.289322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:22.291815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:22.292060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.292750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:56:22.292750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.292750 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:22.292750 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.292750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.292750 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:22.292750 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.292750 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.293780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.294148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:22.294537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.294537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:22.294885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.295873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:22.295873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.295873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:22.308503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:22.309113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:22.309431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:22.309655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:22.310184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:22.310587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.311477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:56:22.311477 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.311477 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:22.311477 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.311477 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.311477 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:22.311477 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.311477 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:22.313114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:22.313807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.313807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:22.313807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.313807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:22.313807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.313807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.313807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:22.313807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.313807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:22.314593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.314593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:22.314593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:22.314593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:22.314593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.314593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:22.314593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:22.314593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:22.314593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:22.314593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.314593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:22.315137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.315719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.315719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:22.315719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.315719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.316137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:22.316988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.317502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.317502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:22.317502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.317502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:22.320644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:22.321036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:22.321171 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:22.321384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:22.321669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:22.321903 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:22.322042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.322560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:56:22.322560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.322560 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:22.322560 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.322560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.322560 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:22.322560 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.322560 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.323436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.323755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:22.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.324089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:22.324434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:22.324867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:22.325397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:22.325397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:22.325397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:27.237513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:27.238603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:27.238603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:27.239519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:27.263659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:27.264287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:27.264515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:27.266391 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:27.266777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:27.267126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.269199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:27.269199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.269199 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:27.269199 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.269199 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.269199 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:27.269199 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.269199 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.273558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.274016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:27.274428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.274428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:27.276672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.277181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.277767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:27.277767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.277767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:27.280500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:27.280916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.281739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:27.281739 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.281739 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:27.281739 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.281739 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.281739 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:27.281739 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.281739 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.282960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.283333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:27.283801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.283801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:27.284569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.285068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.285637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:27.285637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.285637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:27.285637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:27.285637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:27.285637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:27.285637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:27.286096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:27.286438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.287594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:27.287594 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.287594 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:27.287594 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.287594 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.287594 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:27.287594 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.287594 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:27.288380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:27.288877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.288877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:27.288877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.288877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:27.288877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.288877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.288877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:27.288877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.288877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.289199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:27.289601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.289986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.289986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:27.289986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.289986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:27.290718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.291281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:27.291771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.291771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:27.291771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.291771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.291771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:27.291771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.291771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:27.309083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:27.309699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:27.310524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:27.310524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:27.310986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:27.311557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.315248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.316091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:27.316903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.316903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:27.316903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:27.316903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:27.316903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.316903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:27.316903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:27.316903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:27.316903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:27.316903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.316903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:27.317337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.317711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.317711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:27.317711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.317711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.318981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:27.319503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.320089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.320089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:27.320089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.320089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:27.321021 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:27.321559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:27.321859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:27.322109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:27.322613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:27.323016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:27.323312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.324171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 13:56:27.324171 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.324171 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:27.324171 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.324171 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.324171 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:27.324171 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.324171 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.328500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.328981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:27.329437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.329437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:27.330137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:27.330631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:27.331254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:27.331254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:27.331254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:32.238161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:32.239295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:32.239295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:32.240298 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:32.264239 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:32.265743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:32.266107 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:32.267113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:32.267331 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:32.267468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.268711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:32.268711 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.268711 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:32.268711 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.268711 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.268711 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:32.268711 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.268711 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.271233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.271734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:32.272325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.272325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:32.276328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.276912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.277562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:32.277562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.277562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:32.280796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:32.281019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.281757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:32.281757 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.281757 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:32.281757 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.281757 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.281757 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:32.281757 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.281757 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.282508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.282923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:32.283310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.283310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:32.283828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.284330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.284885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:32.284885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.284885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:32.285461 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:32.286050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:32.286277 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:32.286491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:32.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:32.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.287696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:56:32.287696 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.287696 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:32.287696 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.287696 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.287696 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:32.287696 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.287696 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:32.289154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:32.289819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.289819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:32.289819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.289819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:32.289819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.289819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.289819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:32.289819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.289819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.290556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:32.290974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.291371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.291371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:32.291371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.291371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:32.292557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.293068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:32.293592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.293592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:32.293592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.293592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.293592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:32.293592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.293592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:32.309731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:32.311154 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:32.311154 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:32.311154 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:32.311404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:32.311656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.313348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:56:32.313348 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.313348 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:32.313348 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.313348 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.313348 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:32.313348 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.313348 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:32.313776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:32.314264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.314264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:32.314264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.314264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:32.314264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.314264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.314264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:32.314264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.314264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:32.314951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.314951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:32.314951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:32.314951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:32.314951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.314951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:32.314951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:32.314951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:32.314951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:32.314951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.314951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:32.315311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.315713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.315713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:32.315713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.315713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.316718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:32.317176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.317659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.317659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:32.317659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.317659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:32.321557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:32.322121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:32.322377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:32.322653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:32.322920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:32.323135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:32.323304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.323866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:56:32.323866 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.323866 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:32.323866 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.323866 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.323866 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:32.323866 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.323866 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.325313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:32.325952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.325952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:32.326997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:32.327447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:32.327925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:32.327925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:32.327925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:37.239026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:37.239854 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:37.240058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:37.241318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:37.267016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:37.267826 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:37.269017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:37.273305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:37.273670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:37.274676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.278227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:37.278227 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.278227 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:37.278227 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.278227 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.278227 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:37.278227 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.278227 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.280326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.280839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:37.281463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.281463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:37.284652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.285797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.287006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:37.287006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.287006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:37.287006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:37.287329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:37.287806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:37.288511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:37.289097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:37.289539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.290432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:37.290432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.290432 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:37.290432 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.290432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.290432 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:37.290432 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.290432 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.291667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.292260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:37.292690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.292690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:37.292690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:37.293096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.293761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.294336 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:37.295804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:37.296344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.296344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:37.296344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.296344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:37.296344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.296344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.296344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:37.296344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.296344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.297146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:37.297502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.297894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.297894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:37.297894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.297894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:37.299033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.299651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:37.300202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.300202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:37.300202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.300202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.300202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:37.300202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.300202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:37.310861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:37.312378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:37.312625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:37.312815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:37.313420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:37.313808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.314746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:56:37.314746 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.314746 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:37.314746 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.314746 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.314746 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:37.314746 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.314746 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:37.315954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:37.317022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.317022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:37.317022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.317022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:37.317022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.317022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.317022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:37.317022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.317022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:37.318426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.318426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:37.318426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:37.318426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:37.318426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.318426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:37.318426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:37.318426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:37.318426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:37.318426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.318426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:37.319301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.319818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.319818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:37.319818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.319818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.321074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:37.321752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.322429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.322429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:37.322429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.322429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:37.322429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:37.323010 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:37.323682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:37.324164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:37.324811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:37.325196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:37.325454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.326354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:56:37.326354 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.326354 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:37.326354 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.326354 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.326354 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:37.326354 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.326354 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.328278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:37.328968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.328968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:37.330035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:37.330618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:37.331168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:37.331168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:37.331168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:42.239935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:42.241340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:42.241340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:42.241695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:42.243449 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:56:42.267813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:42.269672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:42.271589 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:42.272997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:42.273472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:42.273782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.276406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:42.276406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.276406 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:42.276406 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.276406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.276406 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:42.276406 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.276406 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.282047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.282975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:42.283929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:42.285062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.286439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:42.286439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.286439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.286439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:42.286439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.286439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:42.287485 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:42.287787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.288576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:42.288576 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.288576 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:42.288576 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.288576 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.288576 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:42.288576 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.288576 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.289861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.290588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:42.291132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.291132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:42.291132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:42.291326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.291985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.292828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:42.292828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.292828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:42.292828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:42.292828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:42.292828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:42.292828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:42.293201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.293851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:42.293851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.293851 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:42.293851 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.293851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.293851 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:42.293851 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.293851 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:42.294625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:42.295613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.296293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.296877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:42.297465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.298205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:42.298745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.298745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:42.298745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.298745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.298745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:42.298745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.298745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:42.311439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:42.312430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:42.312430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:42.312430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:42.312972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:42.313367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.314153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:56:42.314153 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.314153 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:42.314153 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.314153 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.314153 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:42.314153 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.314153 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:42.315549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:42.316017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.316017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:42.316017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.316017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:42.316017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.316017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.316017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:42.316017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.316017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:42.316899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.316899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:42.316899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:42.316899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:42.316899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.316899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:42.316899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:42.316899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:42.316899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:42.316899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.316899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:42.317282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.317692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.317692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:42.317692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.317692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.318618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:42.319119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.319663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.319663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:42.319663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.319663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:42.322571 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:42.323537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:42.323537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:42.323537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:42.323990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:42.324390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:42.324653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.325454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:56:42.325454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.325454 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:42.325454 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.325454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.325454 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:42.325454 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.325454 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.328192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.328634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:42.328975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:42.329439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:42.330114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:42.330114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:42.330114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:42.330114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:42.330114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:42.330114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:47.240567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:47.241483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:47.241483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:47.242859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:47.269995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:47.271877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:47.272185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:47.274897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:47.276017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:47.276017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.279018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:47.279018 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.279018 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:47.279018 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.279018 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.279018 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:47.279018 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.279018 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.281195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.281605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:47.282005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.282005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:47.283085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.283621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.284121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:47.284121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.284121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:47.289457 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:47.289917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.290878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:47.290878 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.290878 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:47.290878 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.290878 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.290878 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:47.290878 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.290878 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.292427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.292994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:47.293638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.293638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:47.293638 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:47.294076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.294798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.295728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:47.295728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.295728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:47.295728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:47.295728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:47.295728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:47.295728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:47.296075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.296709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:56:47.296709 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.296709 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:47.296709 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.296709 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.296709 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:47.296709 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.296709 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:47.298114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:47.298814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.298814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:47.298814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.298814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:47.298814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.298814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.298814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:47.298814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.298814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.299769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:47.300201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.300650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.300650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:47.300650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.300650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:47.301802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.302323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:47.302938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.302938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:47.302938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.302938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.302938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:47.302938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.302938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:47.311953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:47.313164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:47.313481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:47.313732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:47.314504 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:47.314896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.315959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:56:47.315959 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.315959 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:47.315959 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.315959 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.315959 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:47.315959 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.315959 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:47.316930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:47.317974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:47.318630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:47.319187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:47.319749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.320528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:47.323133 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:47.323721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:47.323969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:47.324462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:47.324779 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:47.325011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:47.325164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:47.327939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:47.328450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.328847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:47.329261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:47.329857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:47.329857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:47.329857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:47.329857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:47.329857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:47.329857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:47.329857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:47.329857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:47.329857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:47.329857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:52.241240 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:52.241911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:52.242083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:52.243478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:52.271877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:52.272594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:52.272953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:52.275780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:52.276378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.278002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:52.278002 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.278002 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:52.278002 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.278002 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.278002 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:52.278002 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.278002 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.279358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.280913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:52.281732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:52.282547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.283708 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.284814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.285222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:52.285635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:52.286080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.287432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:52.287432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.287432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.287432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:52.287432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.287432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:52.293513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:52.293979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:52.294187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:52.294426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:52.294909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:52.295203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.295944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:56:52.295944 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.295944 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:52.295944 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.295944 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.295944 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:52.295944 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.295944 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:52.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:52.297741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.297741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:52.297741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.297741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:52.297741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.297741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.297741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:52.297741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.297741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.298564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:52.298945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.299356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.299356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:52.299356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.299356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:52.300229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.300681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:52.301253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.301253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:52.301253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.301253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.301253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:52.301253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.301253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:52.312492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:52.313380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:52.313602 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:52.313849 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:52.314529 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:52.314928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.315837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:56:52.315837 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.315837 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:52.315837 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.315837 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.315837 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:52.315837 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.315837 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:52.317570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:52.318058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.318058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:52.318058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.318058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:52.318058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.318058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.318058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:52.318058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.318058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:52.319200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.319200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:52.319200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:52.319200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:52.319200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.319200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:52.319200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:52.319200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:52.319200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:52.319200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.319200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:52.319582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.319938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.319938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:52.319938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.319938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.321081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:52.321622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.322114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.322114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:52.322114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.322114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:52.323711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:52.324551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:52.324761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:52.325074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:52.325712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:52.326079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:52.326360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.327144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 13:56:52.327144 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.327144 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:52.327144 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.327144 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.327144 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:52.327144 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.327144 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.328848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.329258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:52.329591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.329591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:52.330617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:52.331324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:52.332152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:52.332152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:52.332152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:57.241930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:57.247017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:57.247017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:57.247017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:57.274176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:57.274962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:57.275480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:57.277884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:57.278392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:57.278725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.280332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:56:57.280332 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.280332 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:57.280332 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.280332 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.280332 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:57.280332 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.280332 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.281689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.282722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:57.283531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.283531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:57.283802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.285088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.285649 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.286625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.286988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:57.287326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.287326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:57.287965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.289016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.289972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:57.289972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.289972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:57.294053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:57.294549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:57.294844 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:57.295349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:56:57.295715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:57.295936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.296606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:56:57.296606 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.296606 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:57.296606 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.296606 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.296606 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:57.296606 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.296606 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:57.298130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:57.298603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.298603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:57.298603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.298603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:57.298603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.298603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.298603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:57.298603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.298603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.299811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:57.300171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.300562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.300562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:57.300562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.300562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:57.301849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.302330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:57.302893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.302893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:57.302893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.302893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.302893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:57.302893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.302893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:57.313189 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:57.313788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:57.314050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:57.314361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:57.315031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:57.315560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.316567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Traceback (most recent call last):
Apr 21 13:56:57.316567 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.316567 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:57.316567 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.316567 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.316567 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:57.316567 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.316567 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:56:57.318190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:56:57.319109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.319109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:56:57.319109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.319109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:56:57.319109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.319109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.319109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:56:57.319109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.319109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:56:57.320305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.320305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:57.320305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:57.320305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:57.320305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.320305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:57.320305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:57.320305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:57.320305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:57.320305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.320305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:57.322405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-802dc66e-cda7-4eed-b8e2-4626366db08c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:57.323394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:57.323744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:56:57.324173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:56:57.324764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:56:57.324957 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:56:57.325283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:56:57.326082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:57.326756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:56:57.326994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.327603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:56:57.327603 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.327603 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:56:57.327603 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.327603 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.327603 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:56:57.327603 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.327603 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.329043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.329656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:56:57.330180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.330180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:56:57.330660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:56:57.331465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:56:57.331942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:56:57.331942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:56:57.331942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:02.242681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:02.243403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:02.243403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:02.244700 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:02.448356 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:02.456134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:02.456316 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:02.456397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:02.456713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:02.459000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:02.460137 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:02.460352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:02.460509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.462032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:02.462480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:02.464370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:02.464498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:02.464653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:02.464878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:02.465549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:57:02.465549 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.465549 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:02.465549 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.465549 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.465549 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:02.465549 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.465549 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.468665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.469460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.470261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:02.471088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.472079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.472079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:02.472079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.472079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:02.472079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:02.472616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:02.472988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.473696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:02.475396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:57:02.475396 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.475396 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:02.475396 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.475396 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.475396 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:02.475396 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.475396 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:02.476088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:02.476352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:02.482102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:02.482933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.482933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:02.482933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.482933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:02.482933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.482933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.482933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:02.482933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.482933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:02.483968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.487887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.019 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.487887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:02.487887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:02.487887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:02.487887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.487887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:02.487887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:02.487887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:02.487887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:02.487887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.487887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:02.488496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.489049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.489049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:02.489049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.489049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:02.489049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:02.489839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:57:02.489839 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.489839 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:02.489839 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.489839 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.489839 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:02.489839 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.489839 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:02.490794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:02.491316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.491316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:02.491316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.491316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:02.491316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.491316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.491316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:02.491316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.491316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:02.491316 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.493905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:02.494413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.495295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.495295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:02.495295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.495295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task set_fip_distributed_flag failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.495693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:02.496337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.496728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.496728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:02.496728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.496728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:02.496728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:02.500939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.501434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.501948 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_mac_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.503790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.504183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:02.504685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.504685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:02.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.507856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.508601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:02.508601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.508601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:02.510191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:02.510486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.511068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Traceback (most recent call last):
Apr 21 13:57:02.511068 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.511068 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:02.511068 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.511068 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.511068 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:02.511068 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.511068 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] OVN maintenance task check_fdb_aging_settings failed after 0.268 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.512082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.512711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d92a746d-e5ac-40e7-8359-75510c55d1a7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:02.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:02.514168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:02.515097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:02.515097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:02.515097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:02.515097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:02.515097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:02.515097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:07.243244 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:07.243797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:07.243932 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:07.245276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:07.280398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:07.280870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.281836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:57:07.281836 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.281836 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:07.281836 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.281836 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.281836 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:07.281836 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.281836 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.282656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.283034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:07.283432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:07.283893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.284502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:07.284502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.284502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.284502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:07.284502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.284502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:07.457105 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:07.458017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:07.458144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:07.459797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:07.460054 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:07.460218 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.460912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:57:07.460912 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.460912 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:07.460912 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.460912 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.460912 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:07.460912 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.460912 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.462647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.463238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:07.463773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.463773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.464901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.465498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:07.466003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.466003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:07.466003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:07.466003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:07.466003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:07.466177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:07.466297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:07.466613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:07.466750 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:07.466914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:07.467297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:07.467942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:07.468197 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:07.468516 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:07.468878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:07.469100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.469741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:57:07.469741 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.469741 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:07.469741 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.469741 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.469741 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:07.469741 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.469741 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:07.470805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:07.471190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.471190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:07.471190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.471190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:07.471190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.471190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.471190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:07.471190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.471190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:07.471190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.473056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:07.473427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.473821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.473821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:07.473821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.473821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:07.473821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:07.476423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.476867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.477346 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:07.477760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:07.478420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:07.478807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.478807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:07.478807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.478807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:07.478807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.478807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.478807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:07.478807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.478807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:07.478807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.480011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.480011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:07.480011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:07.480011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:07.480011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.480011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:07.480011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:07.480011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:07.480011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:07.480011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.480011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:07.480356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.480718 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.481627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:07.482083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.482685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.482685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:07.482685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.482685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_mac_aging_settings failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.483615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.483946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:07.484294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.484294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:07.485376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:07.485806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:07.486339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:07.486339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:07.486339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:12.244074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:12.244981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:12.245186 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:12.246591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:12.282644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:12.283082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.284104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:57:12.284104 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.284104 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:12.284104 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.284104 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.284104 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:12.284104 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.284104 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.285070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.285742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:12.286270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:12.287460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.288412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:12.288412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.288412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.288412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:12.288412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.288412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:12.458373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:12.458802 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:12.459427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:12.460062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:12.460365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:12.460527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.461157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:57:12.461157 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.461157 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:12.461157 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.461157 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.461157 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:12.461157 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.461157 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.462763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.463424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:12.464014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.464014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.464601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.465298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:12.465825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.465825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:12.465947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:12.466678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:12.467013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:12.467313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:12.467527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:12.467948 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:12.468122 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:12.468373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:12.468824 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:12.469013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.469984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:57:12.469984 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.469984 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:12.469984 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.469984 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.469984 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:12.469984 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.469984 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:12.470864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:12.471443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.471443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:12.471443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.471443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:12.471443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.471443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.471443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:12.471443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.471443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:12.471940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:12.472433 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.473878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:12.474302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.474770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.474770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:12.474770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.474770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:12.476487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.476965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:12.477465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.477465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:12.477465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.477465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.477465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:12.477465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.477465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:12.477465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.477782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:12.478127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:57:12.478127 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.478127 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:12.478127 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.478127 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.478127 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:12.478127 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.478127 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:12.478421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:12.479933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:12.480395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.480395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:12.480395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.480395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:12.480395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.480395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.480395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:12.480395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.480395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:12.480395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:12.481928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.481928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:12.481928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:12.481928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:12.481928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.481928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:12.481928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:12.481928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:12.481928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:12.481928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.481928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:12.482290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.482658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.482658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:12.482658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.482658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:12.482658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:12.484705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.485194 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.486236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:12.486986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.486986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:12.487279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:12.487754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:12.488325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:12.488325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:12.488325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:17.245090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:17.246096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:17.246186 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:17.247630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:17.285202 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:17.285705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.286815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:57:17.286815 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.286815 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:17.286815 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.286815 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.286815 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:17.286815 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.286815 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.287795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.288467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:17.289103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:17.290034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.290915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:17.290915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.290915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.290915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:17.290915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.290915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:17.459327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:17.460129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:17.460306 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:17.461513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:17.461859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:17.462044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.463079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:57:17.463079 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.463079 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:17.463079 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.463079 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.463079 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:17.463079 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.463079 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.464654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.465255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:17.465804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.465804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:17.466592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.467318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.467879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:17.467879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.467879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:17.467879 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:17.467879 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:17.468034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:17.469383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:17.469383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:17.469769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:17.470195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:17.470418 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:17.470909 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:17.471173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.472258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:57:17.472258 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.472258 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:17.472258 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.472258 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.472258 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:17.472258 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.472258 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:17.473595 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:17.474378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:17.474572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:17.474690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:17.474960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:17.475096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:17.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.478693 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:17.479743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task set_fip_distributed_flag failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.480124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:17.480624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.481007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.481007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:17.481007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.481007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:17.481360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:17.481817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.481817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:17.481817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.481817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:17.481817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.481817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.481817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:17.481817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.481817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:17.482384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.482384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:17.482384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:17.482384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:17.482384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.482384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:17.482384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:17.482384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:17.482384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:17.482384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.482384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:17.482852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.483427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.483427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:17.483427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.483427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:17.483949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.484594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:17.485136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.485136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:17.485136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.485136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.485136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:17.485136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.485136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:17.485136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.485746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:17.486357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Traceback (most recent call last):
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.486889 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] OVN maintenance task update_mac_aging_settings failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.487551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.487902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:17.488283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.488283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f31fc4ea-19de-4ba4-b2f4-0d66d682d30c None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:17.488796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:17.489265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:17.489800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:17.489800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:17.489800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:22.245801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:22.246773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:22.246914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:22.248303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:22.283110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:22.283635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.284713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:57:22.284713 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.284713 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:22.284713 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.284713 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.284713 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:22.284713 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.284713 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.285855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.286312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:22.286752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.286752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:22.286922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.287523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.288146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:22.288146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.288146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:22.460189 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:22.460582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:22.460807 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:22.462257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:22.462805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:22.463078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.464156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Traceback (most recent call last):
Apr 21 13:57:22.464156 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.464156 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:22.464156 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.464156 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.464156 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:22.464156 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.464156 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.465799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.466443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:22.467127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.467127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:22.467707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.468544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.469549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:22.469549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.469549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:22.469549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:22.469549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:22.469549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:22.469549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:22.469549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:22.470024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.470095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:57:22.470095 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.470095 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:22.470095 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.470095 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.470095 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:22.470095 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.470095 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:22.471629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:22.472160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.472160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:22.472160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.472160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:22.472160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.472160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.472160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:22.472160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.472160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:22.472160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.473072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:22.473482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.473857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.473857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:22.473857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.473857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:22.473857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:22.474510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:22.475519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.475519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:22.475519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.475519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.475519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:22.475519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.475519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:22.475519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:22.475519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:22.475795 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:22.475949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:22.476280 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:22.476556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:22.477044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:22.477471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.478498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:57:22.478498 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.478498 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:22.478498 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.478498 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.478498 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:22.478498 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.478498 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:22.480020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:22.480618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.480618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:22.480618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.480618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:22.480618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.480618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.480618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:22.480618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.480618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:22.480618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:22.481536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.481536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:22.481536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:22.481536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:22.481536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.481536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:22.481536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:22.481536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:22.481536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:22.481536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.481536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:22.481902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.482294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.482294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:22.482294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.482294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:22.482294 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.483138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:22.483672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.484183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.484183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:22.484183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.484183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:22.484183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.484472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:57:22.484472 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.484472 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:22.484472 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.484472 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.484472 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:22.484472 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.484472 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task update_mac_aging_settings failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.485892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.486267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:22.486693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.486693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:22.487508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:22.488031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:22.488559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:22.488559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:22.488559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:27.246560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:27.247489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:27.247489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:27.248589 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:27.279073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:27.279520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.280392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:57:27.280392 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.280392 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:27.280392 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.280392 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.280392 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:27.280392 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.280392 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.281230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.281898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:27.282629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:27.283435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.284845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:27.284845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.284845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.284845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:27.284845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.284845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:27.460673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:27.461123 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:27.461205 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:27.462056 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:27.462299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:27.462424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.462898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:57:27.462898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.462898 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:27.462898 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.462898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.462898 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:27.462898 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.462898 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.463907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.464438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.465002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:27.465725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.466650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.466650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:27.466650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.466650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:27.468662 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:27.469281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:27.469410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:27.469547 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:27.469985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:27.470352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.470764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:57:27.470764 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.470764 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:27.470764 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.470764 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.470764 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:27.470764 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.470764 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:27.471621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:27.472420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.473102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.473712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:27.474420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.475040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:27.475403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.475403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:27.475403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.475403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.475403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:27.475403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.475403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:27.475403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:27.475403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:27.475403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:27.475630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:27.475630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:27.475630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.475965 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:27.476174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:57:27.476174 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.476174 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:27.476174 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.476174 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.476174 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:27.476174 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.476174 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:27.476952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:27.477245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.477245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:27.477245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.477245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:27.477245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.477245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.477245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:27.477245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.477245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:27.478099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.478099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:27.478099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:27.478099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:27.478099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.478099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:27.478099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:27.478099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:27.478099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:27.478099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.478099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:27.478354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.478620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.478620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:27.478620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.478620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:27.478620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.479628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:27.480005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.480371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.480371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:27.480371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.480371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:27.480371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:27.480371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:27.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:27.480671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:27.480824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.482268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:57:27.482268 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.482268 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:27.482268 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.482268 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.482268 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:27.482268 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.482268 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.483254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.483503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:27.483737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.483737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:27.484411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:27.484718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:27.485104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:27.485104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:27.485104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:32.247255 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:32.248138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:32.248469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:32.249924 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:32.285203 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:32.285710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.286771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:57:32.286771 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.286771 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:32.286771 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.286771 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.286771 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:32.286771 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.286771 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.287875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.288653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:32.289394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:32.290201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.291966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:32.291966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.291966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.291966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:32.291966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.291966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:32.461406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:32.461855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:32.461947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:32.462994 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:32.463395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:32.463654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.464521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:57:32.464521 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.464521 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:32.464521 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.464521 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.464521 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:32.464521 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.464521 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.465575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.466169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.466757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:32.467417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.468349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.468349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:32.468349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.468349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:32.469400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:32.470233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:32.470538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:32.470770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:32.471429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:32.471733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.472747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:57:32.472747 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.472747 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:32.472747 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.472747 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.472747 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:32.472747 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.472747 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:32.473854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:32.474409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.474409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:32.474409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.474409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:32.474409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.474409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.474409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:32.474409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.474409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.475022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:32.475432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.475927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.475927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:32.475927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.475927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:32.475927 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:32.476976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.477676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:32.478359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.478359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:32.478359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.478359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.478359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:32.478359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.478359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:32.478359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:32.478359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:32.478359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:32.478661 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:32.478661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.479229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:57:32.479229 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.479229 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:32.479229 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.479229 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.479229 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:32.479229 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.479229 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:32.480350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:32.480960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.480960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:32.480960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.480960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:32.480960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.480960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.480960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:32.480960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.480960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:32.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:32.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:32.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:32.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:32.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:32.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:32.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:32.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.481695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:32.482155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.482660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.482660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:32.482660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.482660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:32.482660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.483627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:32.484273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.484785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.484785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:32.484785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.484785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:32.484785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:32.484785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:32.484785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:32.485084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:32.485393 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:32.485673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.486505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:57:32.486505 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.486505 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:32.486505 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.486505 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.486505 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:32.486505 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.486505 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.487757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.488124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:32.488532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.488532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:32.488906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:32.489381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:32.489907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:32.489907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:32.489907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:37.248003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:37.248971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:37.248971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:37.250559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:37.300183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.301380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.301969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:37.302602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.303541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:37.461957 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:37.462678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:37.462902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:37.463914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:37.464231 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:37.464461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.465177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:57:37.465177 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.465177 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:37.465177 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.465177 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.465177 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:37.465177 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.465177 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.466877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.467593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:37.468138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.468138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.468857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.469658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:37.470474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.470474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:37.470474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:37.470706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:37.470910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:37.471106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:37.471708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:37.472031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.472891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Traceback (most recent call last):
Apr 21 13:57:37.472891 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.472891 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:37.472891 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.472891 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.472891 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:37.472891 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.472891 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:37.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:37.474935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.474935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:37.474935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.474935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:37.474935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.474935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.474935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:37.474935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.474935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.476292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:37.476643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.477033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.477033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:37.477033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.477033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:37.478479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.479082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:37.479707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.479707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:37.479707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.479707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.479707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:37.479707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.479707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:37.479707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:37.479707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:37.479707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:37.480082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:37.480369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:37.480715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.481405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:57:37.481405 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.481405 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:37.481405 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.481405 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.481405 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:37.481405 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.481405 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:37.482150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:37.482606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.482606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:37.482606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.482606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:37.482606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.482606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.482606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:37.482606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.482606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:37.483072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.483072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:37.483072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:37.483072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:37.483072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.483072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:37.483072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:37.483072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:37.483072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:37.483072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.483072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:37.483369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.483808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.483808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:37.483808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.483808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.484226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:37.484731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.485348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.485348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:37.485348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.485348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:37.485348 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:37.486198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:37.486441 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:37.486767 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:37.487374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:37.487801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:37.488322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.489241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:57:37.489241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.489241 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:37.489241 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.489241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.489241 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:37.489241 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.489241 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.490994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.491462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:37.491801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.491801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:37.492901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:37.493375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:37.493867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:37.493867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:37.493867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:42.248806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:42.249680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:42.249680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:42.250215 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:42.251635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:57:42.284445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:42.284925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.285921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:57:42.285921 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.285921 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:42.285921 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.285921 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.285921 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:42.285921 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.285921 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.286908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.287351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:42.287767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.287767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:42.287961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.288951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.290066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:42.290066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.290066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:42.462604 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:42.463139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:42.463274 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:42.464165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:42.464598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:42.464854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.465842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:57:42.465842 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.465842 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:42.465842 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.465842 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.465842 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:42.465842 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.465842 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.466630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.467152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.467693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:42.468375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.469156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.469156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:42.469156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.469156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:42.470736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:42.471224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:42.471449 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:42.471655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:42.472056 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:42.472282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.472957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:57:42.472957 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.472957 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:42.472957 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.472957 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.472957 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:42.472957 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.472957 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:42.474386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:42.475687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.476877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.477776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:42.478614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.480501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:42.481097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.481097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:42.481097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.481097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.481097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:42.481097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.481097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:42.481097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:42.481097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:42.481097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:42.481508 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:42.481508 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:42.481508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Traceback (most recent call last):
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.484449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:42.485485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.486487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.487154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:42.488256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:42.489713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.491697 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.492476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.493141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:42.493831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:42.494305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:42.494805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:42.494805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:42.494805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:42.494805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:42.494805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:42.494805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:47.249595 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:47.250522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:47.250615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:47.251976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:47.295529 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:47.295952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.296985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:57:47.296985 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.296985 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:47.296985 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.296985 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.296985 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:47.296985 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.296985 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.298000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.298695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:47.299382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:47.300170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.301174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:47.301174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.301174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.301174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:47.301174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.301174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:47.463458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:47.464012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:47.464199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:47.465244 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:47.465604 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:47.465841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.466742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:57:47.466742 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.466742 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:47.466742 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.466742 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.466742 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:47.466742 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.466742 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.468229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.468847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:47.469441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.469441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:47.469955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.470760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.471615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:47.471615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.471615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:47.471615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:47.471615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:47.471877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:47.471877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:47.472299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:47.472558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.473229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:57:47.473229 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.473229 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:47.473229 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.473229 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.473229 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:47.473229 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.473229 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:47.474492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:47.474960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.474960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:47.474960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.474960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:47.474960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.474960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.474960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:47.474960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.474960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.475358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:47.475745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.476116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.476116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:47.476116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.476116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:47.476445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.476898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:47.477408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.477408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:47.477408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.477408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.477408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:47.477408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.477408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:47.480318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:47.480778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:47.480983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:47.481204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:47.481763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:47.482097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.482855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:57:47.482855 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.482855 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:47.482855 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.482855 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.482855 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:47.482855 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.482855 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:47.484248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:47.484769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.484769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:47.484769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.484769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:47.484769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.484769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.484769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:47.484769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.484769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:47.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:47.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:47.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:47.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:47.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:47.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:47.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:47.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:47.486271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:47.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.486636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.488724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:47.489281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.489902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.489902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:47.489902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.489902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:47.489902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:47.489902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:47.489902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:47.489902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:47.490174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:47.490477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:47.490732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.491410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:57:47.491410 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.491410 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:47.491410 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.491410 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.491410 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:47.491410 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.491410 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.491991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.492377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:47.492749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:47.493173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:47.493719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:47.493719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:47.493719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:47.493719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:47.493719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:47.493719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:52.250371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:52.251339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:52.251339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:52.252489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:52.281320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:52.281573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.282366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:57:52.282366 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.282366 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:52.282366 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.282366 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.282366 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:52.282366 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.282366 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.283944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.284957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:52.285838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:52.286806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.287958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:52.287958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.287958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.287958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:52.287958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.287958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:52.464066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:52.464702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:52.464927 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:52.466190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:52.466587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:52.466840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.467751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:57:52.467751 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.467751 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:52.467751 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.467751 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.467751 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:52.467751 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.467751 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.469246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.469886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:52.470509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.470509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:52.470963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.472048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.473506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:52.473506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.473506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:52.473506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:52.473506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:52.473506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:52.473506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:52.473506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:52.474197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:57:52.474483 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.474483 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:52.474483 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.474483 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.474483 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:52.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.474483 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:52.475232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:52.475881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.475881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:52.475881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.475881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:52.475881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.475881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.475881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:52.475881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.475881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.476320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:52.476838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:52.477377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.477953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.478766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:52.480749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:52.481847 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:52.482053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:52.482305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:52.482786 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:52.483109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.483782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Traceback (most recent call last):
Apr 21 13:57:52.483782 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.483782 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:52.483782 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.483782 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.483782 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:52.483782 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.483782 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:52.485122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:52.485920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.485920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:52.485920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.485920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:52.485920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.485920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.485920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:52.485920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.485920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:52.486762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.486762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:52.486762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:52.486762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:52.486762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.486762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:52.486762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:52.486762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:52.486762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:52.486762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.486762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:52.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.487826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.487826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:52.487826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.487826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.487989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:52.488705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.489352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.489352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:52.489352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.489352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:52.489352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:52.489642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:52.489750 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:52.489915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:52.490264 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:52.490535 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:52.490721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.491319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:57:52.491319 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.491319 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:52.491319 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.491319 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.491319 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:52.491319 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.491319 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.492200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.492568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:52.492941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.492941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:52.493452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:52.493946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:52.494467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:52.494467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:52.494467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:57.251017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:57.251649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:57.251811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:57.253144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:57.291514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:57.292165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.293184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:57:57.293184 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.293184 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:57.293184 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.293184 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.293184 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:57.293184 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.293184 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.294553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.295294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:57.296291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:57.297299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.298371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:57.298371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.298371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.298371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:57.298371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.298371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:57.464712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:57.465496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:57.465768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:57.467369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:57.467799 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:57.468136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.469159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:57:57.469159 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.469159 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:57.469159 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.469159 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.469159 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:57.469159 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.469159 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.470755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.471470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:57.472147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.472147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:57.472586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.473732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.475168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:57.475168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.475168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:57.475168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:57.475168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:57.475168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:57.475168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:57:57.475168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:57.475795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.475985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:57:57.475985 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.475985 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:57.475985 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.475985 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.475985 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:57.475985 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.475985 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:57.477437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:57.478008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.478008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:57.478008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.478008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:57.478008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.478008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.478008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:57.478008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.478008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.478822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:57.479249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.479617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.479617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:57.479617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.479617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:57.480227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.480694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:57.481236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.481236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:57.481236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.481236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.481236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:57.481236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.481236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:57.481920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:57.482522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:57.482750 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:57.483002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:57.483526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:57.483837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.484519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Traceback (most recent call last):
Apr 21 13:57:57.484519 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.484519 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:57.484519 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.484519 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.484519 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:57.484519 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.484519 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:57:57.485870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:57:57.486399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.486399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:57:57.486399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.486399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:57:57.486399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.486399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.486399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:57:57.486399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.486399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:57:57.487419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.487419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:57.487419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:57.487419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:57.487419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.487419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:57.487419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:57.487419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:57.487419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:57.487419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.487419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:57.487784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.488170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.488170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:57.488170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.488170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.488683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:57.489302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.490026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.490026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:57.490026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.490026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:57:57.490026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:57:57.490026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:57:57.490026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:57:57.490353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:57:57.490483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:57.490697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:57:57.490801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.491223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:57:57.491223 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.491223 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:57:57.491223 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.491223 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.491223 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:57:57.491223 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.491223 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.492079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.492632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:57:57.493119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:57:57.493574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:57:57.494124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:57:57.494124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:57:57.494124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:57:57.494124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:57:57.494124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:57:57.494124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:02.251561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:02.252446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:02.252610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:02.254547 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:02.296272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:02.296272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.296802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:02.296802 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.296802 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:02.296802 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.296802 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.296802 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:02.296802 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.296802 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.298132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.298654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:02.299136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.299136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:02.299566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.300191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.301363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:02.301363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.301363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:02.465245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:02.465553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:02.465553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:02.466315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:02.466446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:02.466597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.467077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Traceback (most recent call last):
Apr 21 13:58:02.467077 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.467077 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:02.467077 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.467077 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.467077 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:02.467077 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.467077 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.467930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.468230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:02.468593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.468593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-4b945033-e878-4165-a0b1-f75e0ccc5c60 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:02.468659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.469000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.469378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:02.469378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.469378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:02.473381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:02.474048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:02.474314 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:02.474499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:02.475029 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.478249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.478867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.479294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:02.479583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.479857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.479857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:02.479857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.479857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:02.481001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.481441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:02.481857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.481857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:02.481857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.481857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.481857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:02.481857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.481857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:02.483193 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:02.484550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:02.484677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:02.484820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:02.485372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:02.485728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.486483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Traceback (most recent call last):
Apr 21 13:58:02.486483 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.486483 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:02.486483 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.486483 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.486483 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:02.486483 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.486483 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:02.487633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:02.488202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.488202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:02.488202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.488202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:02.488202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.488202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.488202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:02.488202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.488202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:02.489288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.489288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:02.489288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:02.489288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:02.489288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.489288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:02.489288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:02.489288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:02.489288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:02.489288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.489288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:02.489754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.490175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.490175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:02.490175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.490175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:02.491680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.492289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.492289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:02.492289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.492289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:02.492289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:02.492289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:02.492289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:02.492289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:02.492806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:02.493146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:02.493414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.494102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Traceback (most recent call last):
Apr 21 13:58:02.494102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.494102 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:02.494102 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.494102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.494102 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:02.494102 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.494102 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.494978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.495561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:02.496018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.496018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-d932ce75-6067-4f5b-b241-bcdb9f229c12 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:02.497229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:02.497878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:02.500623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:02.500623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:02.500623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:07.254119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:07.254960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:07.254960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:07.256280 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:07.295682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:07.296144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.297155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:07.297155 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.297155 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:07.297155 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.297155 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.297155 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:07.297155 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.297155 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.297919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.298203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:07.298452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.298452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:07.298759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.299130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.299521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:07.299521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.299521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:07.465793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:07.466104 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:07.466307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:07.467252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:07.467516 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:07.467697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.468376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Traceback (most recent call last):
Apr 21 13:58:07.468376 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.468376 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:07.468376 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.468376 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.468376 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:07.468376 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.468376 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.471349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.471887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:07.472232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:07.472693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.473328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:07.473328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.473328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.473328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:07.473328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.473328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:07.473784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:07.474168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:07.474274 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:07.474333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:07.474675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:07.474835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.475226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:58:07.475226 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.475226 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:07.475226 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.475226 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.475226 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:07.475226 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.475226 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:07.475905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.476262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:07.476262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.476262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:07.476262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.476262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.476262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:07.476262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.476262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.476495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:07.476848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.477137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.477137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:07.477137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.477137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.477241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:07.477596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.478056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.478056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:07.478056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.478056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:07.484551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:07.485419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:07.485419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:07.485419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:07.485859 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:07.486114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Traceback (most recent call last):
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.490240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:07.491237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.491782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.492354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:07.492860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.492860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.493153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:07.493872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.494701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.494701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:07.494701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.494701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:07.495645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:07.496126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:07.496258 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:07.496391 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:07.496905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:07.497154 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:07.497478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.497997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:58:07.497997 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.497997 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:07.497997 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.497997 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.497997 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:07.497997 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.497997 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.498918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.499195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:07.499522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.499522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:07.499687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:07.500056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:07.500707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:07.500707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:07.500707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:12.254733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:12.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:12.255616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:12.256844 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:12.297104 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:12.300471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.300471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:12.300471 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.300471 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:12.300471 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.300471 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.300471 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:12.300471 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.300471 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.301530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.301892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:12.302301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.302301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:12.302620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.303097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.303639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:12.303639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.303639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:12.466412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:12.466989 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:12.467158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:12.468244 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:12.468504 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:12.468696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.469443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Traceback (most recent call last):
Apr 21 13:58:12.469443 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.469443 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:12.469443 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.469443 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.469443 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:12.469443 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.469443 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.470951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.471498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:12.472096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.472096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a6811d36-9e1b-466b-af36-8da291638bb1 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:12.472830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.473533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.474321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:12.474321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.474321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:12.474783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:12.475371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:12.475534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:12.475754 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:12.476454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:12.476902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.477836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:58:12.477836 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.477836 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:12.477836 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.477836 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.477836 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:12.477836 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.477836 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:12.479239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.480043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:12.480043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.480043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:12.480043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.480043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.480043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:12.480043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.480043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.480746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:12.481078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.481502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.481502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:12.481502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.481502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.481939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:12.482489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.483021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.483021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:12.483021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.483021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:12.484899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:12.485477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:12.485692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:12.485915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:12.486352 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:12.486869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.487471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:58:12.487471 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.487471 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:12.487471 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.487471 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.487471 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:12.487471 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.487471 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:12.488911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:12.489392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.489392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:12.489392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.489392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:12.489392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.489392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.489392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:12.489392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.489392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:12.490180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.490180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:12.490180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:12.490180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:12.490180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.490180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:12.490180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:12.490180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:12.490180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:12.490180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.490180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:12.490506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.490877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.490877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:12.490877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.490877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.491706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:12.492162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.492675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.492675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:12.492675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.492675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:12.496815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:12.497363 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:12.497576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:12.497782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:12.498267 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:12.498703 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:12.500961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.502161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Traceback (most recent call last):
Apr 21 13:58:12.502161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.502161 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:12.502161 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.502161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.502161 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:12.502161 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.502161 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.504479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.505393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:12.505942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.505942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9ce18373-f466-4b9d-97c9-a0bc2945b009 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:12.506166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:12.517734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:12.519933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:12.519933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:12.519933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:17.255281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:17.256448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:17.256448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:17.257940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:17.291645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:17.291947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.293312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:17.293312 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.293312 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:17.293312 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.293312 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.293312 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:17.293312 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.293312 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.294007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.294929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:17.295566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:17.296185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.296944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:17.296944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.296944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.296944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:17.296944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.296944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:17.467200 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:17.468011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:17.468787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:17.470126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:17.470620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:17.471028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.472120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:58:17.472120 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.472120 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:17.472120 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.472120 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.472120 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:17.472120 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.472120 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.474546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.475528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:17.476477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:17.477416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.477991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:17.477991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.477991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.477991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:17.477991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.477991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:17.477991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:17.477991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:17.477991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:17.477991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:17.478388 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:17.478388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.478607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:58:17.478607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.478607 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:17.478607 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.478607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.478607 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:17.478607 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.478607 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:17.479622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.480452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:17.480452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.480452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:17.480452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.480452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.480452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:17.480452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.480452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.480852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:17.481592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.482225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.482225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:17.482225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.482225 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.482433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:17.482904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.483502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.483502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:17.483502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.483502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:17.485401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:17.485758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:17.485891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:17.486031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:17.486431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:17.486661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.487495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:58:17.487495 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.487495 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:17.487495 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.487495 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.487495 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:17.487495 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.487495 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:17.488592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:17.489042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.489042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:17.489042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.489042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:17.489042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.489042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.489042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:17.489042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.489042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:17.490723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.490723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:17.490723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:17.490723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:17.490723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.490723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:17.490723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:17.490723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:17.490723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:17.490723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.490723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:17.491095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.491519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.491519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:17.491519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.491519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.493114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:17.493696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.494277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.494277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:17.494277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.494277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:17.497427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:17.497992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:17.498231 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:17.498660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:17.499114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:17.499538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:17.499771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.500719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Traceback (most recent call last):
Apr 21 13:58:17.500719 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.500719 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:17.500719 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.500719 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.500719 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:17.500719 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.500719 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.503316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.503785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:17.504167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.504167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0529f655-f325-49e6-8173-6d6e5486f577 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:17.505502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:17.506567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:17.507485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:17.507485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:17.507485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:22.256044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:22.256895 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:22.256895 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:22.258079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:22.294111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:22.294594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.295450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:22.295450 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.295450 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:22.295450 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.295450 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.295450 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:22.295450 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.295450 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.296294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.296908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:22.297750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:22.298448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.299001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:22.299001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.299001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.299001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:22.299001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.299001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:22.468116 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:22.468657 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:22.468868 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:22.470455 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:22.470828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:22.471074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.472307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Traceback (most recent call last):
Apr 21 13:58:22.472307 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.472307 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:22.472307 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.472307 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.472307 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:22.472307 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.472307 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.473835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.474514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:22.475063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.475063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-59dc6769-20d9-4d22-b215-6996953c101d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:22.475777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.476598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.477707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:22.477707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.477707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:22.477707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:22.477956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:22.477956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:22.478093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:22.478587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:22.478958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.479398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:58:22.479398 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.479398 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:22.479398 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.479398 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.479398 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:22.479398 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.479398 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:22.480279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:22.480704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.480704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:22.480704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.480704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:22.480704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.480704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.480704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:22.480704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.480704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.481110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:22.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.481806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.481806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:22.481806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.481806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:22.482135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.482591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:22.483017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.483017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:22.483017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.483017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.483017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:22.483017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.483017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:22.486063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:22.486552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:22.487225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:22.487536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:22.487939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:22.488224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.488841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:58:22.488841 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.488841 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:22.488841 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.488841 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.488841 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:22.488841 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.488841 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:22.492299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:22.492895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.492895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:22.492895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.492895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:22.492895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.492895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.492895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:22.492895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.492895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:22.493298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.493298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:22.493298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:22.493298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:22.493298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.493298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:22.493298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:22.493298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:22.493298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:22.493298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.493298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:22.493819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.494176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.494176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:22.494176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.494176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.495609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:22.496136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.496978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.496978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:22.496978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.496978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:22.497955 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:22.498721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:22.498871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:22.499156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:22.499782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:22.500066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:22.500265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.501000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Traceback (most recent call last):
Apr 21 13:58:22.501000 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.501000 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:22.501000 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.501000 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.501000 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:22.501000 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.501000 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.502566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.503299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-6e7800a3-22ea-4fdd-8c80-ec8f58cb0fd2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:22.504182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:22.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:22.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:22.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:22.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:22.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:22.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:22.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:27.256740 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:27.257625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:27.257625 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:27.258925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:27.295369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:27.295988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.296838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:27.296838 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.296838 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:27.296838 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.296838 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.296838 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:27.296838 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.296838 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.297678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.298406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:27.299080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:27.299890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.300845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:27.300845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.300845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.300845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:27.300845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.300845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:27.468826 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:27.469226 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:27.469351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:27.470017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:27.470269 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:27.470499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.471195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Traceback (most recent call last):
Apr 21 13:58:27.471195 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.471195 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:27.471195 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.471195 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.471195 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:27.471195 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.471195 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.472196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.472577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:27.473011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.473011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:27.473334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.473952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.474686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:27.474686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.474686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:27.477705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:27.478293 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:27.478538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:27.478716 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:27.479262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:27.479472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.480070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:58:27.480070 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.480070 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:27.480070 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.480070 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.480070 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:27.480070 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.480070 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:27.481457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:27.481936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.481936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:27.481936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.481936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:27.481936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.481936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.481936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:27.481936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.481936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.482919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:27.483322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.483662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.483662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:27.483662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.483662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:27.484483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.484890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:27.485402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.485402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:27.485402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.485402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.485402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:27.485402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.485402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:27.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:27.487362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:27.487578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:27.487890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:27.488552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:27.489014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.489801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:58:27.489801 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.489801 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:27.489801 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.489801 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.489801 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:27.489801 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.489801 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:27.491065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:27.491507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.491507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:27.491507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.491507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:27.491507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.491507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.491507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:27.491507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.491507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:27.492589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.492589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:27.492589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:27.492589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:27.492589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.492589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:27.492589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:27.492589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:27.492589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:27.492589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.492589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:27.492914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.493297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.493297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:27.493297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.493297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.494456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:27.494963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.495535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.495535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:27.495535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.495535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:27.498559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:27.498871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:27.499012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:27.499182 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:27.499503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:27.499886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:27.500126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.500845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:58:27.500845 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.500845 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:27.500845 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.500845 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.500845 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:27.500845 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.500845 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.501720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.502064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:27.502436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.502436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:27.502697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:27.503110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:27.503622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:27.503622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:27.503622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:32.257519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:32.258503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:32.258503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:32.259958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:32.295783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:32.296180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.297081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:32.297081 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.297081 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:32.297081 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.297081 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.297081 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:32.297081 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.297081 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.298085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.298763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:32.299472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:32.300340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.301349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:32.301349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.301349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.301349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:32.301349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.301349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:32.469543 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:32.470302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:32.470457 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:32.472010 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:32.472533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:32.472859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.474001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Traceback (most recent call last):
Apr 21 13:58:32.474001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.474001 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:32.474001 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.474001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.474001 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:32.474001 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.474001 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.475851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.476670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9d37f6af-55fd-4544-b665-79b4a7ad48be None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:32.477531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:32.478513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.479648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:32.479648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.479648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.479648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:32.479648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.479648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:32.479648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:32.479648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:32.479648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:32.479648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:32.480087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:32.480260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.480956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:58:32.480956 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.480956 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:32.480956 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.480956 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.480956 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:32.480956 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.480956 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:32.482356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:32.482947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.482947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:32.482947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.482947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:32.482947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.482947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.482947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:32.482947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.482947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.483740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:32.484125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.484588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.484588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:32.484588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.484588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:32.485430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.485887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:32.486351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.486351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:32.486351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.486351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.486351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:32.486351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.486351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:32.487075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:32.487572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:32.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:32.487980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:32.488476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:32.488743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.489404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:58:32.489404 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.489404 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:32.489404 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.489404 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.489404 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:32.489404 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.489404 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:32.490711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:32.491395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.491395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:32.491395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.491395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:32.491395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.491395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.491395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:32.491395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.491395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:32.492293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.492293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:32.492293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:32.492293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:32.492293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.492293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:32.492293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:32.492293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:32.492293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:32.492293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.492293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:32.492657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.493045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.493045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:32.493045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.493045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.494007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:32.494514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.495084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.495084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:32.495084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.495084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:32.499176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:32.499793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:32.499946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:32.500169 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:32.500544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:32.500825 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:32.501004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.501615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:58:32.501615 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.501615 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:32.501615 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.501615 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.501615 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:32.501615 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.501615 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.502627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.503032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:32.503425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.503425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:32.503718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:32.504455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:32.505340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:32.505340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:32.505340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:37.258261 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:37.259628 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:37.259628 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:37.259990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:37.295351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:37.295624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.296336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:37.296336 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.296336 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:37.296336 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.296336 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.296336 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:37.296336 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.296336 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.297054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.297552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:37.298017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:37.298552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.299194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:37.299194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.299194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.299194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:37.299194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.299194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:37.470668 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:37.470997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:37.471104 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:37.471858 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:37.472037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:37.472338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.473070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Traceback (most recent call last):
Apr 21 13:58:37.473070 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.473070 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:37.473070 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.473070 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.473070 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:37.473070 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.473070 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.473986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.474559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:37.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.475012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a4be10a4-33ed-4849-9760-8c05583851b7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:37.475271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.475775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.476530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:37.476530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.476530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:37.478849 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:37.479670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:37.479783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:37.479884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:37.480297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:37.480503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.480977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:58:37.480977 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.480977 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:37.480977 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.480977 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.480977 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:37.480977 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.480977 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:37.481731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:37.482869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.483384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.483822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:37.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.484749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:37.485482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.485482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:37.485482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.485482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.485482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:37.485482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.485482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:37.487558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:37.488236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:37.488395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:37.488584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:37.489099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:37.489364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.489904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:58:37.489904 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.489904 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:37.489904 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.489904 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.489904 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:37.489904 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.489904 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:37.491332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.492101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:37.492101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.492101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:37.492101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.492101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.492101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:37.492101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.492101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:37.492759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.493373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.493950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:37.493950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.493950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.494567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:37.495323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.496043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.496043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:37.496043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.496043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:37.499828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:37.500386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:37.500582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:37.500886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:37.501498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:37.501972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:37.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.503408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Traceback (most recent call last):
Apr 21 13:58:37.503408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.503408 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:37.503408 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.503408 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.503408 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:37.503408 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.503408 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.505003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:37.505754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.505754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-05325dbd-aa0d-4647-9430-9c7382a6883a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:37.507493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:37.507930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:37.508473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:37.508473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:37.508473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:42.261268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:42.261268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:42.261268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:42.261268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:42.268773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:58:42.297359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:42.297827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.298884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:42.298884 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.298884 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:42.298884 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.298884 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.298884 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:42.298884 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.298884 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.299893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.300152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:42.300399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.300399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:42.300996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.301365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.301704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:42.301704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.301704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:42.471030 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:42.473282 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:42.473554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:42.474714 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:42.475166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:42.475451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.477550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:58:42.477550 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.477550 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:42.477550 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.477550 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.477550 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:42.477550 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.477550 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.478801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.479328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:42.479656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.479656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:42.480222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.480706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.481246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:42.481246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.481246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:42.481246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:42.481246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:42.481246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:42.481431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:42.482389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:42.482654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.483367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:58:42.483367 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.483367 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:42.483367 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.483367 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.483367 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:42.483367 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.483367 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:42.484717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:42.485074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.485074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:42.485074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.485074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:42.485074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.485074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.485074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:42.485074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.485074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.486293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:42.486588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.486853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.486853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:42.486853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.486853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:42.488289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.488874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:42.489290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.489290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:42.489290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.489290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.489290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:42.489290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.489290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:42.489290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:42.489290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:42.489290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:42.489290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:42.489524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:42.489994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.490454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:58:42.490454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.490454 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:42.490454 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.490454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.490454 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:42.490454 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.490454 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:42.491064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.491423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:42.491423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.491423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:42.491423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.491423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.491423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:42.491423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.491423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:42.492025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.492327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:42.492661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:42.492990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.493361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:42.493361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.493361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:42.493361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.493361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.493361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:42.493361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.493361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:42.500298 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:42.500835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:42.501793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:42.501793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:42.501910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:42.502354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:42.502519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.503850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:58:42.503850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.503850 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:42.503850 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.503850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.503850 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:42.503850 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.503850 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.504931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.505200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:42.505444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.505444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:42.506028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:42.506353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:42.506691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:42.506691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:42.506691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:47.272606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:47.272606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:47.272606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:47.272606 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:47.300492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:47.300666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.301432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:47.301432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.301432 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:47.301432 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.301432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.301432 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:47.301432 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.301432 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.302399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.302869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:47.303604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.303604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:47.303865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.304356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.304901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:47.304901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.304901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:47.473323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:47.473694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:47.473773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:47.474571 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:47.474773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:47.474979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.475492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:58:47.475492 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.475492 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:47.475492 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.475492 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.475492 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:47.475492 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.475492 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.476281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.476572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:47.476856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.476856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:47.480496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.480937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.481323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:47.481323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.481323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:47.481323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:47.481963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:47.482198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:47.482526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:47.483161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:47.483368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.484090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:58:47.484090 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.484090 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:47.484090 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.484090 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.484090 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:47.484090 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.484090 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:47.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:47.486011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.486011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:47.486011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.486011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:47.486011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.486011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.486011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:47.486011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.486011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.487425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:47.489075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:47.489362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.489697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:47.490119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:47.490402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:47.490655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:47.490655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:47.491010 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:47.491226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.491765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:58:47.491765 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.491765 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:47.491765 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.491765 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.491765 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:47.491765 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.491765 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:47.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.493485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:47.493485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.493485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:47.493485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.493485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.493485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:47.493485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.493485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:47.493989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.494266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.494538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:47.494538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.494538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.495094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:47.495651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.496132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.496132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:47.496132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.496132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:47.501816 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:47.502099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:47.502248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:47.502484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:47.502809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:47.503015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:47.503138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.505459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:58:47.505459 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.505459 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:47.505459 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.505459 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.505459 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:47.505459 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.505459 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.506310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.506675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:47.506973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.506973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:47.508170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:47.508616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:47.509023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:47.509023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:47.509023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:52.260796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:52.261712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:52.261953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:52.263578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:52.321783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:52.322165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.325120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:52.325120 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.325120 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:52.325120 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.325120 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.325120 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:52.325120 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.325120 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.062 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.326867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.327468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:52.327855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.327855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:52.329425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.330082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.330892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:52.330892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.330892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:52.473896 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:52.476247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:52.476247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:52.476247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:52.476397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:52.476577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.477593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:58:52.477593 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.477593 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:52.477593 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.477593 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.477593 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:52.477593 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.477593 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.478945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.479281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:52.479558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.479558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.480477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.480884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:52.481259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.481259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:52.482815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:52.482997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:52.483186 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:52.483418 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:52.484200 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:52.484724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.485932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:58:52.485932 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.485932 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:52.485932 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.485932 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.485932 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:52.485932 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.485932 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:52.487091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:52.487666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.487666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:52.487666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.487666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:52.487666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.487666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.487666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:52.487666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.487666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.489351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:52.489983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:52.491951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.492545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:52.493120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:52.493599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:52.493599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:52.493599 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:52.493599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.496298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.496961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:52.499245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.499245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:52.499245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:52.499245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:52.499245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.499245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:52.499245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:52.499245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:52.499245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:52.499245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.499245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:52.499701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:52.500248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:52.500704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.501303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:52.501303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:52.501303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.501303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:52.501303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.501303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:52.501303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.501303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.501303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:52.501303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.501303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:52.502617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:52.503245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:52.503336 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:52.503515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:52.504064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:52.504444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:52.504636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.505193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Traceback (most recent call last):
Apr 21 13:58:52.505193 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.505193 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:52.505193 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.505193 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.505193 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:52.505193 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.505193 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.506114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.506558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:52.506917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.506917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-97f34696-5f00-46a1-8b51-76347f34d691 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:52.507292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:52.507670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:52.508192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:52.508192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:52.508192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:57.261657 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:57.262395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:57.262395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:57.263096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:57.298663 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:57.298918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.299638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:58:57.299638 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.299638 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:57.299638 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.299638 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.299638 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:57.299638 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.299638 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.300566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.300995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:57.301385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.301385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:57.301711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.303070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.303634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:57.303634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.303634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:57.474692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:57.475402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:57.475491 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:57.477401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:57.477401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:57.477401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.477401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:58:57.477401 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.477401 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:57.477401 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.477401 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.477401 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:57.477401 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.477401 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.477930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.478279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:57.478964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.478964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.479728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.480836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:57.482132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.482132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:57.483162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:57.483624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:57.483701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:57.483792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:58:57.484232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:57.484421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.484958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:58:57.484958 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.484958 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:57.484958 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.484958 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.484958 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:57.484958 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.484958 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:57.485929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:57.486779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.486779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:57.486779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.486779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:57.486779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.486779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.486779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:57.486779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.486779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.487244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:57.487947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:57.488680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.489557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.490548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:57.491253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:57.491561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:57.491677 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:57.491811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:57.493687 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:57.493960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.495691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:58:57.495691 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.495691 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:57.495691 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.495691 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.495691 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:57.495691 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.495691 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:58:57.496570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:58:57.497132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.497132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:58:57.497132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.497132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:58:57.497132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.497132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.497132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:58:57.497132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.497132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:58:57.497689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.497689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:57.497689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:57.497689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:57.497689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.497689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:57.497689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:57.497689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:57.497689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:57.497689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.497689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:57.498028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:57.498557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:57.499099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.499634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:57.499634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:57.499634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.499634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:57.499634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.499634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:57.499634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.499634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.499634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:57.499634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.499634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:58:57.503301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:58:57.503920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:58:57.504033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:58:57.504578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:58:57.505136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:57.505439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:58:57.505637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.509406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:58:57.509406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.509406 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:58:57.509406 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.509406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.509406 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:58:57.509406 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.509406 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.510292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:58:57.511140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:58:57.511767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:58:57.512659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:58:57.514007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:58:57.514007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:58:57.514007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:02.262521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:02.263506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:02.263506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:02.264569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:02.296901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:02.297234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.297820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:02.297820 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.297820 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:02.297820 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.297820 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.297820 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:02.297820 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.297820 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.298651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.299401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:02.300113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:02.300983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.302063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:02.302063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.302063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.302063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:02.302063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.302063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:02.475936 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:02.476370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:02.476512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:02.477521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:02.478018 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:02.478326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.479114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:02.479114 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.479114 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:02.479114 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.479114 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.479114 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:02.479114 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.479114 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.480144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.480411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:02.480806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.480806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.481396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.481747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:02.482104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.482104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:02.483605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:02.484569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:02.484633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:02.484725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:02.485328 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:02.485617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.486301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:02.486301 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.486301 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:02.486301 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.486301 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.486301 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:02.486301 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.486301 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:02.486955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.487763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:02.488560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.489147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:02.489742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.490624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:02.491803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:02.491911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:02.492044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:02.492177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:02.492717 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:02.493197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.494025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:59:02.494025 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.494025 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:02.494025 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.494025 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.494025 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:02.494025 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.494025 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:02.494905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:02.495571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.495571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:02.495571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.495571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:02.495571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.495571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.495571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:02.495571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.495571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:02.495795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.495795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:02.495795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:02.495795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:02.495795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.495795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:02.495795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:02.495795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:02.495795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:02.495795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.495795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:02.496114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.496468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.496468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:02.496468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.496468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.496866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:02.497337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.497821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.497821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:02.497821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.497821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:02.503930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:02.504189 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:02.504307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:02.504458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:02.504993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:02.505424 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:02.505711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.506455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Traceback (most recent call last):
Apr 21 13:59:02.506455 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.506455 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:02.506455 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.506455 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.506455 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:02.506455 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.506455 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.507341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.507928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7c55ed31-8e71-4478-8b1f-63e48a01c62d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:02.508580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:02.509279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:02.510159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:02.510159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:02.510159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:02.510159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:02.510159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:02.510159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:07.263035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:07.263694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:07.263694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:07.264417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:07.290807 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:07.291185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.291983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:07.291983 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.291983 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:07.291983 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.291983 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.291983 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:07.291983 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.291983 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.292726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.293060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:07.293363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.293363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:07.293481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.293782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.294138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:07.294138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.294138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:07.476473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:07.477347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:07.477464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:07.478686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:07.478912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:07.479072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.479681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:07.479681 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.479681 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:07.479681 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.479681 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.479681 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:07.479681 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.479681 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.480729 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.481362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.481895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:07.482758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.483600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.483600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:07.483600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.483600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:07.487550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:07.487972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:07.488066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:07.488163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:07.488616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:07.488822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.489360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:07.489360 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.489360 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:07.489360 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.489360 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.489360 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:07.489360 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.489360 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:07.490223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.490648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:07.490648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.490648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:07.490648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.490648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.490648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:07.490648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.490648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.491260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:07.491572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.491961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.491961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:07.491961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.491961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.492528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:07.493243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.494037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.494037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:07.494037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.494037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:07.494037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:07.494037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:07.494037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:07.494037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:07.494433 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:07.494433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.495041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:59:07.495041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.495041 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:07.495041 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.495041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.495041 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:07.495041 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.495041 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:07.496050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:07.496687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.496687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:07.496687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.496687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:07.496687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.496687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.496687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:07.496687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.496687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:07.497550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.497550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:07.497550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:07.497550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:07.497550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.497550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:07.497550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:07.497550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:07.497550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:07.497550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.497550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:07.498043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:07.498640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:07.499251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.500031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:07.500031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:07.500031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.500031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:07.500031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.500031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:07.500031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.500031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.500031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:07.500031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.500031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:07.504396 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:07.504913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:07.505041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:07.505264 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:07.505655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:07.505899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:07.508480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.509195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Traceback (most recent call last):
Apr 21 13:59:07.509195 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.509195 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:07.509195 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.509195 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.509195 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:07.509195 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.509195 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.510733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.511318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:07.511865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.511865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-fbc17899-ca08-4d2e-9ac6-f661ed3c4a97 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:07.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:07.518174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:07.519014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:07.519014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:07.519014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:12.263770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:12.264555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:12.264783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:12.266384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:12.302200 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:12.302488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.303060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:12.303060 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.303060 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:12.303060 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.303060 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.303060 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:12.303060 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.303060 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.304017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.304658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:12.305257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:12.306023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.307054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:12.307054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.307054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.307054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:12.307054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.307054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:12.478090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:12.478090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:12.478938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:12.480587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:12.480813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:12.481309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:12.483721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.484505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:12.485043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.485043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.485043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:12.485043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.485043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.486162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.486964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:12.487556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.487556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:12.488020 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:12.488361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:12.488452 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:12.488568 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:12.489669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:12.489900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.490611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:59:12.490611 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.490611 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:12.490611 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.490611 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.490611 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:12.490611 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.490611 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:12.491489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:12.492122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.492544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.492936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:12.493348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.493947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:12.494789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.494789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:12.494789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.494789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.494789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:12.494789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.494789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:12.494789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:12.494789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:12.494789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:12.494789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:12.495306 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:12.495306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.496017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:12.496017 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.496017 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:12.496017 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.496017 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.496017 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:12.496017 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.496017 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:12.496921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.497939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:12.497939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.497939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:12.497939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.497939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.497939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:12.497939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.497939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:12.498328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.498891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.499531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:12.499531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.499531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.502121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:12.502802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.503305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.503305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:12.503305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.503305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:12.504891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:12.505664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:12.505791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:12.506050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:12.506770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:12.507073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:12.507282 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.507961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Traceback (most recent call last):
Apr 21 13:59:12.507961 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.507961 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:12.507961 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.507961 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.507961 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:12.507961 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.507961 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.509202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.509566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:12.509916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.509916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-414eff1b-c4d5-45bd-baec-fc88279cbf15 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:12.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:12.510925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:12.511400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:12.511400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:12.511400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:17.264813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:17.265935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:17.265935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:17.267009 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:17.303467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:17.303467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.304179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:17.304179 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.304179 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:17.304179 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.304179 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.304179 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:17.304179 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.304179 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.305027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.305414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:17.305821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.305821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:17.306014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.306551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.307154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:17.307154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.307154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:17.478252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:17.479101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:17.479446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:17.480671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:17.481092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:17.481364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.482306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Traceback (most recent call last):
Apr 21 13:59:17.482306 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.482306 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:17.482306 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.482306 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.482306 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:17.482306 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.482306 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.483548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.483965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:17.484390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.484390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-7582e666-c293-4892-9321-ec10377a6c67 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:17.484641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.485128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.485720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:17.485720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.485720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:17.488537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:17.493450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:17.494425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:17.494425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:17.494670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:17.495313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:17.495654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.496496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:59:17.496496 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.496496 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:17.496496 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.496496 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.496496 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:17.496496 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.496496 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:17.498202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:17.498924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.498924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:17.498924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.498924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:17.498924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.498924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.498924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:17.498924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.498924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.499958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:17.500581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.501265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.501265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:17.501265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.501265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:17.501742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.502594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:17.503456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.503456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:17.503456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.503456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.503456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:17.503456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.503456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:17.503456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:17.503456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:17.503456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:17.503919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:17.503919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.504610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:59:17.504610 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.504610 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:17.504610 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.504610 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.504610 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:17.504610 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.504610 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:17.505175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:17.505589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.505589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:17.505589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.505589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:17.505589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.505589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.505589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:17.505589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.505589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:17.506193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.506193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:17.506193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:17.506193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:17.506193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.506193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:17.506193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:17.506193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:17.506193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:17.506193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.506193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:17.506512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.506862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.506862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:17.506862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.506862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.506990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:17.507467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.508024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.508024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:17.508024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.508024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:17.508024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:17.509382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:17.509540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:17.509668 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:17.510396 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:17.510807 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:17.510997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.511541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Traceback (most recent call last):
Apr 21 13:59:17.511541 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.511541 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:17.511541 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.511541 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.511541 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:17.511541 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.511541 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.512423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.512794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:17.513188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.513188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-908c7036-6127-4128-aa63-43dbccfa8328 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:17.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:17.513961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:17.514517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:17.514517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:17.514517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:22.265771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:22.267110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:22.267110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:22.268130 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:22.297635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:22.298119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.299197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:22.299197 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.299197 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:22.299197 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.299197 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.299197 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:22.299197 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.299197 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.300371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.301045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:22.301769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:22.302688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.303732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:22.303732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.303732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.303732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:22.303732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.303732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:22.478926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:22.479524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:22.479722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:22.481472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:22.481923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:22.482414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.483440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Traceback (most recent call last):
Apr 21 13:59:22.483440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.483440 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:22.483440 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.483440 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.483440 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:22.483440 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.483440 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.485293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.485900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:22.487118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.487118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-5e2b0fba-58cc-42d8-84b5-7fce55b8f578 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:22.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.488935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.490511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:22.490511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.490511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:22.494025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:22.494965 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:22.495238 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:22.495523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:22.496096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:22.496373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.497109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:59:22.497109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.497109 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:22.497109 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.497109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.497109 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:22.497109 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.497109 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:22.498171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:22.498554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.498554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:22.498554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.498554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:22.498554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.498554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.498554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:22.498554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.498554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.499098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:22.499386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.499655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.499655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:22.499655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.499655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:22.500259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.500717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:22.501204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.501204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:22.501204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.501204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.501204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:22.501204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.501204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:22.503003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:22.503447 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:22.503676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:22.503844 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:22.505289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:22.505411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.506590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:22.506590 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.506590 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:22.506590 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.506590 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.506590 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:22.506590 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.506590 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:22.508076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.508605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:22.508605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.508605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:22.508605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.508605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.508605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:22.508605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.508605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:22.508903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.509289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.509660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:22.509660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.509660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:22.510369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.511342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:22.511863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.512387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.512387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:22.512387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.512387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:22.512387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:22.512387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:22.512387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:22.512699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:22.513037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:22.513231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.514055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Traceback (most recent call last):
Apr 21 13:59:22.514055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.514055 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:22.514055 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.514055 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.514055 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:22.514055 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.514055 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.515943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.516346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:22.516668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.516668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-71cb4465-dc6f-4d42-b464-691559b7e3a2 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:22.517009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:22.517498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:22.517984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:22.517984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:22.517984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:27.266991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:27.267878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:27.268075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:27.269932 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:27.307702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:27.308362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.309486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:27.309486 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.309486 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:27.309486 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.309486 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.309486 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:27.309486 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.309486 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.310649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.311124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:27.311503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.311503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:27.312031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.312525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.313112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:27.313112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.313112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:27.479480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:27.479999 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:27.480138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:27.481124 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:27.481455 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:27.481685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.482614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Traceback (most recent call last):
Apr 21 13:59:27.482614 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.482614 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:27.482614 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.482614 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.482614 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:27.482614 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.482614 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.483655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.484393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f7e107d5-2134-4f01-956c-97efa6ff18a4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:27.485075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:27.485846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.486902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:27.486902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.486902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.486902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:27.486902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.486902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:27.494601 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:27.494898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:27.495016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:27.495135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:27.495634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:27.495862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.496522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:59:27.496522 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.496522 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:27.496522 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.496522 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.496522 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:27.496522 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.496522 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:27.498086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:27.498569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.498569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:27.498569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.498569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:27.498569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.498569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.498569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:27.498569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.498569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.499625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:27.499966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.500373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.500373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:27.500373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.500373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:27.501437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.501938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:27.502491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.502491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:27.502491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.502491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.502491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:27.502491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.502491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:27.503597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:27.504313 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:27.504534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:27.504816 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:27.505307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:27.505541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.506096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:59:27.506096 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.506096 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:27.506096 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.506096 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.506096 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:27.506096 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.506096 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:27.506975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:27.507715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.507715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:27.507715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.507715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:27.507715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.507715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.507715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:27.507715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.507715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:27.508141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.508141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:27.508141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:27.508141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:27.508141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.508141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:27.508141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:27.508141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:27.508141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:27.508141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.508141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:27.508688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:27.509348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:27.510052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:27.510970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:27.511602 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:27.511699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:27.511884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:27.512238 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:27.512524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:27.512642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Traceback (most recent call last):
Apr 21 13:59:27.513349 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.513349 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:27.513349 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.513349 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.513349 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:27.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.513349 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.514279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:27.515024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.515024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-6f6a8c4a-f093-40ba-9b47-0baedd43ba22 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:27.515340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:27.516189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:27.516730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:27.516730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:27.516730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:32.267674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:32.268517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:32.268517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:32.269447 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:32.305759 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:32.305994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.306677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:32.306677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.306677 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:32.306677 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.306677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.306677 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:32.306677 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.306677 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.307574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.308140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:32.308886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:32.309677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.310777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:32.310777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.310777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.310777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:32.310777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.310777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:32.480171 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:32.480976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:32.481090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:32.482149 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:32.482551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:32.482847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.484016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:32.484016 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.484016 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:32.484016 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.484016 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.484016 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:32.484016 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.484016 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.485407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.486014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:32.486699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.486699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.487160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.487695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:32.488183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.488183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:32.495062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:32.495567 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:32.495821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:32.496088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:32.496852 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:32.497146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.498088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:59:32.498088 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.498088 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:32.498088 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.498088 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.498088 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:32.498088 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.498088 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:32.499488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:32.499922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.499922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:32.499922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.499922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:32.499922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.499922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.499922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:32.499922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.499922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.501027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:32.501433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.501789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.501789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:32.501789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.501789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:32.503272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.503698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:32.504172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.504172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:32.504172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.504172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.504172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:32.504172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.504172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:32.504386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:32.504869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:32.505077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:32.505363 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:32.506007 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:32.506598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.507377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:59:32.507377 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.507377 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:32.507377 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.507377 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.507377 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:32.507377 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.507377 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:32.508758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:32.509133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.509133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:32.509133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.509133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:32.509133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.509133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.509133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:32.509133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.509133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:32.510352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.510352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:32.510352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:32.510352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:32.510352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.510352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:32.510352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:32.510352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:32.510352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:32.510352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.510352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:32.510682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.511048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.511048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:32.511048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.511048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.512678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:32.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.513630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.513630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:32.513630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.513630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:32.513630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:32.513630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:32.513630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:32.513855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:32.514594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:32.514841 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:32.515029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.515652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Traceback (most recent call last):
Apr 21 13:59:32.515652 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.515652 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:32.515652 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.515652 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.515652 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:32.515652 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.515652 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.516589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.516921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:32.517280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.517280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-c5bea4ac-3792-4162-9d24-49876977102a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:32.517632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:32.518097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:32.518769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:32.518769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:32.518769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:37.268257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:37.271410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:37.271410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:37.271410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:37.309872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:37.310168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.310898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:37.310898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.310898 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:37.310898 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.310898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.310898 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:37.310898 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.310898 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.312008 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.312468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:37.312947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.312947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:37.313062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.313630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.314323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:37.314323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.314323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:37.481199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:37.481505 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:37.481505 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:37.482510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:37.482835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:37.483077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.483911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:37.483911 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.483911 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:37.483911 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.483911 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.483911 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:37.483911 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.483911 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.485565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.486327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:37.486911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.486911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.488174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.488968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:37.489549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.489549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:37.495480 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:37.495940 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:37.496142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:37.496386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:37.497333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:37.497666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.498722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Traceback (most recent call last):
Apr 21 13:59:37.498722 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.498722 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:37.498722 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.498722 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.498722 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:37.498722 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.498722 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:37.499995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:37.500934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.500934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:37.500934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.500934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:37.500934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.500934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.500934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:37.500934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.500934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.501445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:37.502091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.502774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.502774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:37.502774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.502774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9b5d1f47-c990-4d72-93e0-c48b2c1656d3 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:37.503117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.503962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:37.504915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.504915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:37.504915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.504915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.504915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:37.504915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.504915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:37.504915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:37.505329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:37.505329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:37.505329 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:37.505704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:37.505915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.506421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Traceback (most recent call last):
Apr 21 13:59:37.506421 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.506421 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:37.506421 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.506421 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.506421 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:37.506421 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.506421 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:37.507238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:37.507678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.507678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:37.507678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.507678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:37.507678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.507678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.507678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:37.507678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.507678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:37.508201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.508201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:37.508201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:37.508201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:37.508201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.508201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:37.508201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:37.508201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:37.508201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:37.508201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.508201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:37.508576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.508980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.508980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:37.508980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.508980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ebfb96c5-df3d-4ba2-93c3-3db57a35cf5b None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:37.510293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.510789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.510789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:37.510789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.510789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:37.513561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:37.515151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:37.515426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:37.515985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:37.516355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:37.517265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.517899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Traceback (most recent call last):
Apr 21 13:59:37.517899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.517899 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:37.517899 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.517899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.517899 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:37.517899 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.517899 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.519091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.519977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:37.520622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.520622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-e833837d-fe88-4b56-914d-c51166747bb8 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:37.521973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:37.522762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:37.523614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:37.523614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:37.523614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:42.269701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:42.271378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:42.271378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:42.272468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:42.279911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=241 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 13:59:42.311683 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:42.311888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.312679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:42.312679 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.312679 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:42.312679 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.312679 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.312679 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:42.312679 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.312679 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.314257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.314805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:42.315392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.315392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:42.316059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.316757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.317685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:42.317685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.317685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:42.481854 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:42.482376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:42.482569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:42.483831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:42.484382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:42.484655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.485800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 13:59:42.485800 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.485800 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:42.485800 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.485800 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.485800 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:42.485800 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.485800 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.488120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.488927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:42.489549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.489549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:42.490271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.490988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.491839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:42.491839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.491839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:42.495932 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:42.496347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:42.496469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:42.496598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:42.496951 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:42.497146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.497646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 13:59:42.497646 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.497646 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:42.497646 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.497646 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.497646 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:42.497646 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.497646 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:42.498438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:42.498864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.498864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:42.498864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.498864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:42.498864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.498864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.498864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:42.498864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.498864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.499442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:42.499811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.500182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.500182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:42.500182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.500182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:42.500784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.501231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:42.501719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.501719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:42.501719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.501719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.501719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:42.501719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.501719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:42.505094 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:42.505408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:42.505498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:42.505665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:42.506524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:42.507088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.508175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:59:42.508175 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.508175 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:42.508175 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.508175 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.508175 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:42.508175 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.508175 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:42.509046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:42.509470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.509470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:42.509470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.509470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:42.509470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.509470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.509470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:42.509470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.509470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:42.509972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.509972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:42.509972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:42.509972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:42.509972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.509972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:42.509972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:42.509972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:42.509972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:42.509972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.509972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:42.510310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.510679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.510679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:42.510679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.510679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.511352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:42.511952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.512517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.512517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:42.512517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.512517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:42.513947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:42.514343 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:42.514475 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:42.514696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:42.515129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:42.515422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:42.515586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.516238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:42.516238 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.516238 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:42.516238 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.516238 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.516238 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:42.516238 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.516238 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.517808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:42.518192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.518546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:42.520306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:42.520842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:42.521396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:42.521396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:46.462363 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:46.463013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:46.463013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:46.472467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 21 13:59:46.472605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106267) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 21 13:59:46.472791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Fixing resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:59:46.510534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, resource={'id': 'b57c058f-054e-4a2a-980f-6bfda2111e90', 'name': 'private', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['6a3993e6-b5ba-4812-baa3-b25db5880cd9', 'c97deb27-a1d6-4e03-b489-95abde8fd791'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:29:32Z', 'updated_at': '2026-04-21T13:29:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 24380}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.510946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.511508 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3f6c78fa-b671-412d-bb34-5426a40d8bff, external_ids_update=None, columns={'tag': 24380}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.511700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.512774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:59:46.512774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.512774 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:46.512774 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.512774 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.512774 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:46.512774 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.512774 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:46.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Failed to fix resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:46.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:59:46.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:59:46.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:59:46.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:59:46.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:59:46.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:46.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.515155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.515550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:46.515550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.515550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:46.515550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Fixing resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:59:46.544376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d0607710-681d-444d-be26-dbeca5a07b46, resource={'id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'name': 'public', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-21T13:29:38Z', 'updated_at': '2026-04-21T13:29:48Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.544921 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d0607710-681d-444d-be26-dbeca5a07b46, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.546346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-19f04b40-ed99-482a-8774-a76892a716d1, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.547633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.549607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:59:46.549607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.549607 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:46.549607 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.549607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.549607 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:46.549607 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.549607 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:46.550902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Failed to fix resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.550902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:46.550902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:59:46.550902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:59:46.550902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:59:46.550902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:59:46.550902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:59:46.550902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:46.550902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.550902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.550902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.551324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.551727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:46.551727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.551727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:46.551727 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Fixing resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:59:46.593561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, resource={'id': '345e31fe-ede8-4dd0-823e-e81dcbc5bef0', 'name': 'admin_net', 'tenant_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'project_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['bb271c4b-4172-418f-995c-f72abd439dbe'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:34:46Z', 'updated_at': '2026-04-21T13:34:47Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 22486}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.593822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.594079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-c1b4e94e-49ea-4c1f-8edf-2f1433fdff16, external_ids_update=None, columns={'tag': 22486}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.594159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.594744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:59:46.594744 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.594744 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:46.594744 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.594744 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.594744 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:46.594744 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.594744 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:46.596200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Failed to fix resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.596200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:46.596200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:59:46.596200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:59:46.596200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:59:46.596200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:59:46.596200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:59:46.596200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:46.596200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.596200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.596200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.596791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.597659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:46.597659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.597659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:46.597659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Fixing resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:59:46.635550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, resource={'id': 'b364ad09-f70d-45d8-b9dd-8420f5f1c377', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['620e9559-2c15-4370-904f-f4677bc69542'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:35:24Z', 'updated_at': '2026-04-21T13:35:25Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 61006}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.635981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.636534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a100d4e8-7f11-40c9-8e4a-346c2c764f69, external_ids_update=None, columns={'tag': 61006}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.636711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.637877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:59:46.637877 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.637877 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:46.637877 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.637877 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.637877 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:46.637877 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.637877 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:46.639306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Failed to fix resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.639306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:46.639306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:59:46.639306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:59:46.639306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:59:46.639306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:59:46.639306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 13:59:46.639306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:46.639306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.639306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.639306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.639844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.640593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:46.640593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.640593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:46.640593 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Fixing resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:59:46.725119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, resource={'id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'enable_snat': True}, 'gw_port_id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-61m1f', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'ip_address': '172.24.5.49', 'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'}, {'ip_address': '2001:db8::162', 'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 3, 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04'}, resource_type=routers, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.725523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.725705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.726614 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.726614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.727167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:59:46.727167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.727167 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:46.727167 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.727167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.727167 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:46.727167 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.727167 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:46.727679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Unable to update router 06899d4c-a436-45e4-bed5-95dabe888f51. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.729022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Failed to fix resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.729022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:46.729022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:59:46.729022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:59:46.729022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:59:46.729022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:59:46.729022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 21 13:59:46.729022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 21 13:59:46.729022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 21 13:59:46.729022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 21 13:59:46.729022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:46.729698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:46.730129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:46.730129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:46.730129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.730129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:46.730129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.730129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.730129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:46.730129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.730129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:46.730129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Fixing resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 13:59:46.836517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, resource={'id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'name': '', 'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'project_id': '', 'mac_address': 'fa:16:3e:fb:71:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000168688', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.836769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3 ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'neutron:network_name': 'neutron-d0607710-681d-444d-be26-dbeca5a07b46', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-06899d4c-a436-45e4-bed5-95dabe888f51'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.49/24', '2001:db8::162/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.837415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, lrouter_port=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:46.837794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.838799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Traceback (most recent call last):
Apr 21 13:59:46.838799 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.838799 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:46.838799 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.838799 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.838799 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:46.838799 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.838799 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:46.839780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] Maintenance task: Failed to fix resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.839780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:46.839780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 13:59:46.839780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 13:59:46.839780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 13:59:46.839780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 13:59:46.839780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 21 13:59:46.839780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:46.839780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.839780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.839780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:46.840285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:46.840891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:46.840891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:46.840891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:46.840891 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-607bd9ea-8121-40bc-8b8c-56d101cadedd None None] OVN maintenance task check_for_inconsistencies finished in 0.377 seconds
Apr 21 13:59:47.271265 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:47.271848 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:47.272048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:47.273158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:47.308149 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:47.308537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.309349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:47.309349 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.309349 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:47.309349 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.309349 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.309349 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:47.309349 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.309349 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.310754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.311337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:47.311861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.311861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:47.312194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.312909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.313716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:47.313716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.313716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:47.482464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:47.483387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:47.483387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:47.483951 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:47.484272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:47.484622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.485253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Traceback (most recent call last):
Apr 21 13:59:47.485253 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.485253 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:47.485253 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.485253 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.485253 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:47.485253 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.485253 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.486251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.486862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:47.487496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:47.488366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.489327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:47.489327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.489327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.489327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:47.489327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.489327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:47.496430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:47.496989 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:47.497234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:47.497476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:47.497992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:47.498199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.498890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 13:59:47.498890 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.498890 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:47.498890 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.498890 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.498890 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:47.498890 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.498890 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:47.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:47.501397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.501397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:47.501397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.501397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:47.501397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.501397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.501397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:47.501397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.501397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.501857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:47.502545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.503183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.503183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:47.503183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.503183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:47.503819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.504668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:47.505487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.505487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:47.505487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.505487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.505487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:47.505487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.505487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:47.505487 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:47.505876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:47.505944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:47.506098 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:47.506619 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:47.506980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.507877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:59:47.507877 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.507877 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:47.507877 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.507877 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.507877 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:47.507877 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.507877 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:47.508649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:47.509439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.509439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:47.509439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.509439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:47.509439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.509439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.509439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:47.509439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.509439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:47.509767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.509767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:47.509767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:47.509767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:47.509767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.509767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:47.509767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:47.509767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:47.509767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:47.509767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.509767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:47.510426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:47.510882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:47.511308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.511823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:47.511823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:47.511823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.511823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:47.511823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.511823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:47.511823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.511823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.511823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:47.511823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.511823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:47.514395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:47.514843 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:47.515031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:47.515263 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:47.515981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:47.516479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:47.516710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.517659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:47.517659 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.517659 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:47.517659 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.517659 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.517659 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:47.517659 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.517659 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.519062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:47.519490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.519910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:47.521073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:47.521656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:47.522388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:47.522388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:52.272179 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:52.273037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:52.273037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:52.273847 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:52.310061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:52.310432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.311140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:52.311140 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.311140 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:52.311140 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.311140 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.311140 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:52.311140 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.311140 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.311886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.312552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:52.313171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:52.314007 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.315312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:52.315312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.315312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.315312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:52.315312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.315312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:52.483164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:52.483802 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:52.484012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:52.485163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:52.485458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:52.485626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.486498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Traceback (most recent call last):
Apr 21 13:59:52.486498 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.486498 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:52.486498 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.486498 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.486498 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:52.486498 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.486498 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.488701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.489335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:52.489913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.489913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:52.491567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.492430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.493502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:52.493502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.493502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:52.496920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:52.497769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:52.497769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:52.497769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:52.498177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:52.498502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.499360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 13:59:52.499360 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.499360 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:52.499360 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.499360 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.499360 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:52.499360 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.499360 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:52.500375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:52.501005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.501462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.501838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:52.502158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.502609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:52.503086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.503086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:52.503086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.503086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.503086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:52.503086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.503086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:52.506000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:52.506311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:52.506492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:52.506566 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:52.507000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:52.507426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.508175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:59:52.508175 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.508175 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:52.508175 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.508175 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.508175 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:52.508175 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.508175 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:52.508931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:52.509416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.509416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:52.509416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.509416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:52.509416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.509416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.509416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:52.509416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.509416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:52.509787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.509787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:52.509787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:52.509787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:52.509787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.509787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:52.509787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:52.509787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:52.509787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:52.509787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.509787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:52.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.510609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.510609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:52.510609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.510609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:52.511393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:52.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:52.514930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:52.515447 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:52.515584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:52.515799 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:52.516405 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:52.516775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:52.517014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.517860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:52.517860 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.517860 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:52.517860 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.517860 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.517860 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:52.517860 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.517860 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.519276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:52.519710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.520097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:52.520789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:52.521309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:52.521817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:52.521817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:57.273101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:57.274006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:57.274006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:57.275473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:57.312026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:57.312026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.312310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 13:59:57.312310 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.312310 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:57.312310 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.312310 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.312310 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:57.312310 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.312310 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.313311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.315443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:57.316181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:57.316591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.317059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:57.317059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.317059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.317059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:57.317059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.317059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:57.483914 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:57.484285 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:57.484395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:57.486386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:57.486386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:57.486386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.487019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:57.487019 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.487019 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:57.487019 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.487019 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.487019 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:57.487019 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.487019 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.487657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.488155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:57.488613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.488613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.489027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.489595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:57.490203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.490203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:57.497429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:57.497991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:57.498190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:57.498437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 13:59:57.499054 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:57.499421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.500152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 13:59:57.500152 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.500152 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:57.500152 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.500152 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.500152 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:57.500152 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.500152 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:57.501563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:57.502045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.502045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:57.502045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.502045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:57.502045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.502045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.502045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:57.502045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.502045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.502966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:57.503671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.504431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.504431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:57.504431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.504431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:57.504638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.505436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:57.506198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.506198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:57.506198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.506198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.506198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:57.506198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.506198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:57.506710 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:57.506911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:57.507085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:57.507237 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:57.507581 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:57.507810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.508313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 13:59:57.508313 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.508313 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:57.508313 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.508313 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.508313 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:57.508313 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.508313 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 13:59:57.509041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 13:59:57.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 13:59:57.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 13:59:57.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 13:59:57.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.509771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 13:59:57.510245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.510245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:57.510245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:57.510245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:57.510245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.510245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:57.510245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:57.510245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:57.510245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:57.510245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.510245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:57.510888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:57.511302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:57.511654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.512040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:57.512040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:57.512040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.512040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:57.512040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.512040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:57.512040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.512040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.512040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:57.512040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.512040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 13:59:57.515477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 13:59:57.516050 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 13:59:57.516218 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 13:59:57.516541 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 13:59:57.516893 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:57.517173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 13:59:57.517405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.517989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 13:59:57.517989 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.517989 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 13:59:57.517989 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.517989 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.517989 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 13:59:57.517989 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.517989 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.519573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 13:59:57.520284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.520991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 13:59:57.521176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 13:59:57.522083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 13:59:57.523117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 13:59:57.523117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:02.273969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:02.274986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:02.275087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:02.275876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:02.316525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:02.316931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.318065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:02.318065 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.318065 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:02.318065 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.318065 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.318065 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:02.318065 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.318065 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.319138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.319771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:02.320386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:02.321116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.322024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:02.322024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.322024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.322024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:02.322024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.322024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:02.484522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:02.485094 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:02.485273 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:02.486540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:02.486931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:02.487305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.488240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:02.488240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.488240 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:02.488240 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.488240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.488240 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:02.488240 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.488240 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.490166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.491274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.492089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:02.493176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.494396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.494396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:02.494396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.494396 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:02.497948 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:02.498749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:02.498946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:02.499141 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:02.499621 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:02.499816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.500434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:00:02.500434 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.500434 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:02.500434 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.500434 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.500434 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:02.500434 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.500434 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:02.501773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:02.502203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.502203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:02.502203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.502203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:02.502203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.502203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.502203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:02.502203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.502203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.502902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:02.503233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.503616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.503616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:02.503616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.503616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:02.504013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.504477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:02.504923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.504923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:02.504923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.504923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.504923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:02.504923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.504923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:02.506938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:02.507359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:02.507547 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:02.507769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:02.508187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:02.508549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.509231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:00:02.509231 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.509231 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:02.509231 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.509231 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.509231 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:02.509231 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.509231 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:02.510610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:02.511168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.511168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:02.511168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.511168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:02.511168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.511168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.511168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:02.511168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.511168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:02.512015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.512015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:02.512015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:02.512015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:02.512015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.512015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:02.512015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:02.512015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:02.512015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:02.512015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.512015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:02.512469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.512984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.512984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:02.512984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.512984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.513838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:02.514514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.515188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.515188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:02.515188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.515188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:02.515883 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:02.516260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:02.516357 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:02.516507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:02.516901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:02.517233 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:02.517515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.518270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:02.518270 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.518270 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:02.518270 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.518270 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.518270 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:02.518270 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.518270 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.519691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:02.520181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.520681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:02.520921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:02.522012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:02.522708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:02.522708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:07.275266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:07.276127 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:07.276127 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:07.276127 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:07.313657 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:07.313956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.314752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:07.314752 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.314752 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:07.314752 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.314752 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.314752 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:07.314752 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.314752 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.315787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.316303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:07.316842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:07.317483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.318315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:07.318315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.318315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.318315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:07.318315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.318315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:07.485163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:07.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:07.485646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:07.486723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:07.487043 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:07.487252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.487886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:07.487886 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.487886 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:07.487886 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.487886 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.487886 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:07.487886 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.487886 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.489031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.489607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.490146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:07.490825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.491721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.491721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:07.491721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.491721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:07.498515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:07.499062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:07.499204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:07.499399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:07.500003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:07.500237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.500965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:00:07.500965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.500965 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:07.500965 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.500965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.500965 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:07.500965 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.500965 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:07.502297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:07.502801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.502801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:07.502801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.502801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:07.502801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.502801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.502801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:07.502801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.502801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.503658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:07.504294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:07.505701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.506814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:07.507710 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:07.508696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:07.508696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:07.508696 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:07.509355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:07.509355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.510005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:00:07.510005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.510005 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:07.510005 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.510005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.510005 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:07.510005 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.510005 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:07.511445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:07.512039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.512039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:07.512039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.512039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:07.512039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.512039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.512039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:07.512039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.512039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:07.512952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.512952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:07.512952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:07.512952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:07.512952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.512952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:07.512952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:07.512952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:07.512952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:07.512952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.512952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:07.513476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.514237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.514237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:07.514237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.514237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.514421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:07.514746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.515119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.515119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:07.515119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.515119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:07.516286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:07.516816 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:07.516963 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:07.517188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:07.517835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:07.518093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:07.518268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.518852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:07.518852 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.518852 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:07.518852 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.518852 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.518852 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:07.518852 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.518852 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.520044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:07.520444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.520781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:07.521159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:07.522139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:07.522139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:12.276172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:12.277354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:12.277354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:12.278181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:12.321262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:12.321701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.322663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:12.322663 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.322663 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:12.322663 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.322663 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.322663 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:12.322663 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.322663 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.323626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.324068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:12.324419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:12.324839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.325344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:12.325344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.325344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.325344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:12.325344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.325344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:12.486025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:12.486761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:12.486990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:12.487964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:12.488332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:12.488632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.489537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 14:00:12.489537 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.489537 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:12.489537 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.489537 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.489537 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:12.489537 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.489537 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.490958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.491254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:12.491520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.491520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:12.491997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.492399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.492811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:12.492811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.492811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:12.499374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:12.499672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:12.499794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:12.499960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:12.500678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.504236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.504822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.505357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:12.505715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:12.506105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.506596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.507097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:12.508034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:12.508392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:12.508569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:12.508836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:12.509719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:12.510298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.511459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:00:12.511459 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.511459 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:12.511459 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.511459 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.511459 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:12.511459 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.511459 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:12.512546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:12.513076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.513076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:12.513076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.513076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:12.513076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.513076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.513076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:12.513076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.513076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:12.513658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.513658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:12.513658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:12.513658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:12.513658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.513658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:12.513658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:12.513658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:12.513658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:12.513658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.513658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:12.514164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.514603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.514603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:12.514603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.514603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.515175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:12.515817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.516418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.516418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:12.516418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.516418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:12.516772 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:12.517407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:12.517541 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:12.517818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:12.518414 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:12.518776 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:12.518946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.519634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:12.519634 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.519634 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:12.519634 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.519634 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.519634 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:12.519634 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.519634 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.520847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:12.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.521666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:12.522712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:12.523189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:12.523703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:12.523703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:17.276988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:17.278075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:17.278075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:17.279007 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:17.318028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:17.318807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.319579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:17.319579 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.319579 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:17.319579 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.319579 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.319579 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:17.319579 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.319579 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.320509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.321344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:17.322015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:17.322904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.327941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:17.327941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.327941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.327941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:17.327941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.327941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:17.486905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:17.487877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:17.488047 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:17.488860 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:17.489117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:17.489296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.490287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 14:00:17.490287 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.490287 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:17.490287 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.490287 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.490287 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:17.490287 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.490287 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.492153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.492873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:17.493330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:17.493725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.494178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:17.494178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.494178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.494178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:17.494178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.494178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:17.499772 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:17.500652 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:17.500828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:17.501137 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:17.501784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:17.502089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.502841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:17.502841 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.502841 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:17.502841 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.502841 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.502841 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:17.502841 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.502841 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:17.504122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.504681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:17.504681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.504681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:17.504681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.504681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.504681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:17.504681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.504681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.505391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:17.505886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.506536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.506536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:17.506536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.506536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.506931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:17.507471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.507970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.507970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:17.507970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.507970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:17.508687 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:17.509044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:17.509317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:17.509636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:17.511100 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:17.511503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.512342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:00:17.512342 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.512342 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:17.512342 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.512342 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.512342 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:17.512342 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.512342 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:17.513936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:17.514403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.514403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:17.514403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.514403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:17.514403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.514403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.514403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:17.514403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.514403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:17.515646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.515646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:17.515646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:17.515646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:17.515646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.515646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:17.515646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:17.515646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:17.515646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:17.515646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.515646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:17.516055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.516431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.516431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:17.516431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.516431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.517774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:17.518305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.518780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.518780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:17.518780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.518780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:17.518780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:17.518780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:17.518780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:17.518780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:17.519400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:17.519400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:17.519400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.520105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:17.520105 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.520105 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:17.520105 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.520105 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.520105 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:17.520105 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.520105 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.520930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:17.521524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.521906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:17.522049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:17.523425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:17.524010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:17.524010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:22.278003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:22.278974 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:22.279259 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:22.280330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:22.314954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:22.315403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.316308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:22.316308 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.316308 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:22.316308 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.316308 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.316308 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:22.316308 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.316308 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.317409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.318057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:22.318808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:22.319556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.322563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:22.322563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.322563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.322563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:22.322563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.322563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:22.488637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:22.489199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:22.489404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:22.491873 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:22.492190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:22.492457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.493333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:22.493333 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.493333 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:22.493333 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.493333 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.493333 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:22.493333 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.493333 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.494363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.496284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.497701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:22.498993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.499506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.499506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:22.499506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.499506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:22.500507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:22.500932 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:22.501181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:22.501181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:22.501973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:22.502178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.502944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:00:22.502944 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.502944 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:22.502944 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.502944 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.502944 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:22.502944 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.502944 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:22.504916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:22.505381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.505381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:22.505381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.505381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:22.505381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.505381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.505381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:22.505381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.505381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.506611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:22.506995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.507350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.507350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:22.507350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.507350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:22.508901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.509345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:22.509765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.509765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:22.509765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.509765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.509765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:22.509765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.509765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:22.509765 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:22.510305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:22.510405 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:22.510519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:22.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:22.511452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.512002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:00:22.512002 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.512002 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:22.512002 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.512002 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.512002 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:22.512002 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.512002 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:22.512721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:22.513603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:22.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:22.514693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:22.514986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.515456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:22.518407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:22.518938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:22.519024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:22.519193 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:22.519913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:22.520240 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:22.520412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.521046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:22.521046 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.521046 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:22.521046 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.521046 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.521046 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:22.521046 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.521046 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.522063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:22.522846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:22.523411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:22.524199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:22.525048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:22.525048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:22.525048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:27.279248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:27.280098 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:27.280256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:27.281408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:27.321061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:27.321539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.322462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:27.322462 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.322462 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:27.322462 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.322462 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.322462 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:27.322462 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.322462 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.323204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.323862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:27.324496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:27.325312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.326163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:27.326163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.326163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.326163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:27.326163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.326163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:27.489571 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:27.490507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:27.490634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:27.491778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:27.492010 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:27.492192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.492945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:27.492945 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.492945 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:27.492945 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.492945 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.492945 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:27.492945 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.492945 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.494499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.495068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:27.495679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.495679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.496452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.497205 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:27.497992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.497992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:27.501167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:27.501664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:27.501957 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:27.502162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:27.502833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:27.503054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.503721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:27.503721 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.503721 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:27.503721 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.503721 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.503721 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:27.503721 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.503721 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:27.505302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.505887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:27.505887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.505887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:27.505887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.505887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.505887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:27.505887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.505887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.506863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:27.507382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.507777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.507777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:27.507777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.507777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.508438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:27.509016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.509577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.509577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:27.509577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.509577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:27.510076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:27.510983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:27.510983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:27.510983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:27.511834 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:27.512112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.514493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:27.514493 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.514493 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:27.514493 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.514493 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.514493 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:27.514493 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.514493 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:27.515015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.515604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:27.515604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.515604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:27.515604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.515604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.515604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:27.515604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.515604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:27.515909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.516429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.516842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:27.516842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.516842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:27.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.518434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.518434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:27.518434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.518434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:27.518899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:27.519657 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:27.519867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:27.520128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:27.520543 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:27.520707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:27.520823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.521422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:27.521422 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.521422 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:27.521422 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.521422 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.521422 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:27.521422 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.521422 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.523243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:27.523880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.524469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:27.524616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:27.525359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:27.526125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:27.526125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:32.280377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:32.281109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:32.281109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:32.282390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:32.316546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:32.317372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.318518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:32.318518 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.318518 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:32.318518 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.318518 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.318518 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:32.318518 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.318518 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.319591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.319956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:32.320387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.320387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:32.320726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.321190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.321851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:32.321851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.321851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:32.490473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:32.490838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:32.490924 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:32.492371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:32.492888 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:32.493230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.494348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:32.494348 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.494348 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:32.494348 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.494348 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.494348 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:32.494348 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.494348 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.495851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.496574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:32.497236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.497236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.497547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.498524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:32.499341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.499341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:32.501799 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:32.502434 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:32.502531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:32.502688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:32.503614 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:32.504109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.505149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:32.505149 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.505149 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:32.505149 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.505149 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.505149 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:32.505149 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.505149 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:32.506453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.507257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:32.507257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.507257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:32.507257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.507257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.507257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:32.507257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.507257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.507634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:32.508329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.508954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.508954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:32.508954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.508954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.509152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:32.510001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.510996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.510996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:32.510996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.510996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:32.510996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:32.510996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:32.510996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:32.511386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:32.512342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:32.512863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.513780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:00:32.513780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.513780 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:32.513780 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.513780 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.513780 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:32.513780 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.513780 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:32.514726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:32.515285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.515285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:32.515285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.515285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:32.515285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.515285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.515285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:32.515285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.515285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:32.515606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.515606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:32.515606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:32.515606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:32.515606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.515606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:32.515606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:32.515606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:32.515606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:32.515606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.515606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:32.515903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.516275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.516275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:32.516275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.516275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.516489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:32.516931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.517476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.517476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:32.517476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.517476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:32.519410 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:32.519731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:32.519822 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:32.519980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:32.520734 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:32.521064 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:32.521328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:32.521994 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.521994 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:32.521994 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.521994 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.521994 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:32.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.523565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:32.523907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.524358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:32.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:32.525647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:32.526122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:32.526122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:37.281296 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:37.282283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:37.282283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:37.283339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:37.326656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:37.326928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.328170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:37.328170 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.328170 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:37.328170 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.328170 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.328170 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:37.328170 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.328170 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.329804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.330241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:37.330525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.330525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:37.330948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.331368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.331864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:37.331864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.331864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:37.491478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:37.491867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:37.492121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:37.494309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:37.494817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:37.495136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.496404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:00:37.496404 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.496404 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:37.496404 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.496404 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.496404 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:37.496404 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.496404 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.498121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.498811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:37.499403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.499403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:37.500087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.500981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.501546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:37.501546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.501546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:37.502353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:37.502925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:37.503101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:37.503593 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:37.503801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:37.504048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.504734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:37.504734 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.504734 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:37.504734 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.504734 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.504734 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:37.504734 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.504734 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:37.506297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.506904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:37.506904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.506904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:37.506904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.506904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.506904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:37.506904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.506904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.507695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:37.508104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.508512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.508512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:37.508512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.508512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.509393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:37.509891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.510548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.510548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:37.510548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.510548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:37.511041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:37.511808 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:37.512012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:37.512274 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:37.512982 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:37.513423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.514539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:00:37.514539 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.514539 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:37.514539 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.514539 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.514539 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:37.514539 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.514539 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:37.516006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:37.516498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.516498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:37.516498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.516498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:37.516498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.516498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.516498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:37.516498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.516498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:37.517667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.517667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:37.517667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:37.517667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:37.517667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.517667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:37.517667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:37.517667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:37.517667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:37.517667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.517667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:37.518203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.518710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.518710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:37.518710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.518710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:37.520106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.521333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.521333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:37.521333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.521333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:37.521333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:37.521333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:37.521333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:37.521333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:37.522072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:37.522072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:37.522422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.523180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:37.523180 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.523180 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:37.523180 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.523180 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.523180 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:37.523180 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.523180 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.524263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:37.525043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.525466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:37.525534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:37.526143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:37.526847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:37.526847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:42.281874 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:42.282653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:42.282653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:42.283972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:42.290483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=434 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 14:00:42.318760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:42.318913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.319708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:42.319708 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.319708 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:42.319708 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.319708 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.319708 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:42.319708 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.319708 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.320623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.321178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:42.321597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:42.322002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.322548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:42.322548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.322548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.322548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:42.322548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.322548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:42.492094 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:42.492523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:42.492642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:42.493938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:42.494315 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:42.494555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.495426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:42.495426 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.495426 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:42.495426 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.495426 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.495426 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:42.495426 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.495426 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.497332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.497959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:42.498580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.498580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.500767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.501381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:42.502018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.502018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:42.502877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:42.503471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:42.503539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:42.503617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:42.504034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:42.504352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.504987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:42.504987 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.504987 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:42.504987 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.504987 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.504987 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:42.504987 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.504987 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:42.506475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.506919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:42.506919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.506919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:42.506919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.506919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.506919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:42.506919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.506919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.507922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:42.508268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.508721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.508721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:42.508721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.508721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.509844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:42.510376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.511076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.511076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:42.511076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.511076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:42.511494 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:42.511989 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:42.512135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:42.512515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:42.512969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:42.513499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.514315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:42.514315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.514315 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:42.514315 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.514315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.514315 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:42.514315 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.514315 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:42.515686 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.516154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:42.516154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.516154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:42.516154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.516154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.516154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:42.516154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.516154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:42.517197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.517573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.517953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:42.517953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.517953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.518315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:42.518790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.519352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.519352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:42.519352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.519352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:42.520362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:42.520681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:42.520804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:42.520966 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:42.521298 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:42.521521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:42.521673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.522165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:00:42.522165 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.522165 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:42.522165 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.522165 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.522165 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:42.522165 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.522165 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.523026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.523464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:42.523779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.523779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:42.524001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:42.524501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:42.524972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:42.524972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:42.524972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:47.282637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:47.283360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:47.283578 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:47.285398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:47.316610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:47.316851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.317464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:47.317464 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.317464 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:47.317464 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.317464 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.317464 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:47.317464 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.317464 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.318334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.318945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:47.319654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:47.320513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.321137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:47.321137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.321137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.321137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:47.321137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.321137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:47.492610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:47.492979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:47.493403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:47.494096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:47.494404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:47.494645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.495531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:47.495531 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.495531 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:47.495531 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.495531 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.495531 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:47.495531 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.495531 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.496967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.497601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:47.498317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.498317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.498737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.499707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:47.500252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.500252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:47.503530 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:47.503875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:47.504301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:47.504561 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:47.504915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:47.505095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.505633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:47.505633 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.505633 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:47.505633 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.505633 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.505633 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:47.505633 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.505633 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:47.506898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.507433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:47.507433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.507433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:47.507433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.507433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.507433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:47.507433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.507433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:47.508847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.509343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.509343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:47.509343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.509343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.510031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:47.510569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.511180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.511180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:47.511180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.511180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:47.511947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:47.512523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:47.512699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:47.512876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:47.513478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:47.513852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.514750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:47.514750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.514750 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:47.514750 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.514750 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.514750 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:47.514750 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.514750 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:47.516033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.516656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:47.516656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.516656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:47.516656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.516656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.516656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:47.516656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.516656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:47.517745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.518458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:47.518458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.518458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.519603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:47.520162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.520868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.520868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:47.520868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.520868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:47.520868 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:47.521317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:47.521450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:47.521724 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:47.522099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:47.522618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:47.522720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.523684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:00:47.523684 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.523684 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:47.523684 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.523684 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.523684 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:47.523684 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.523684 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.524908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.525342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:47.525672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.525672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:47.525991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:47.526489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:47.526991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:47.526991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:47.526991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:52.283386 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:52.284172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:52.284172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:52.285043 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:52.316646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:52.316965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.317612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:52.317612 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.317612 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:52.317612 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.317612 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.317612 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:52.317612 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.317612 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.318510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.319104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:52.319728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:52.320430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.321419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:52.321419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.321419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.321419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:52.321419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.321419 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:52.493178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:52.493617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:52.493762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:52.494972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:52.495500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:52.495733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.496719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:52.496719 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.496719 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:52.496719 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.496719 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.496719 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:52.496719 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.496719 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.499115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.499543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:52.499991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.499991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.501289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.501778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:52.502270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.502270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:52.503949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:52.504373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:52.504512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:52.504824 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:52.505230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:52.505611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.506536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 14:00:52.506536 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.506536 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:52.506536 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.506536 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.506536 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:52.506536 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.506536 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:52.507542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:52.508353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.508353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:52.508353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.508353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:52.508353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.508353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.508353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:52.508353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.508353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.508858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:52.509474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.510083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.510083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:52.510083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.510083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:52.510634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.511319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:52.511985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.511985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:52.511985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.511985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.511985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:52.511985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.511985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:52.512406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:52.513065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:52.513065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:52.513065 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:52.513544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:52.513712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:52.516956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.517369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.517680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:52.517923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:52.518180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.518585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:52.518921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.518921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.518921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:52.518921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.518921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:52.521048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:52.521863 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:52.522087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:52.522417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:52.522892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:52.523299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:52.523546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.524423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:00:52.524423 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.524423 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:52.524423 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.524423 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.524423 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:52.524423 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.524423 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.526120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.526537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:52.526899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.526899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:52.527869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:52.528405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:52.528919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:52.528919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:52.528919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:57.284124 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:57.285002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:57.285002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:57.286046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:57.321996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:57.322439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.323316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:00:57.323316 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.323316 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:57.323316 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.323316 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.323316 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:57.323316 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.323316 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.324226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.324733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:57.325297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:57.325885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.326669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:57.326669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.326669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.326669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:57.326669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.326669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:57.493997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:57.495741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:57.495741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:57.496007 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:57.496406 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:57.496685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:57.499748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:57.500195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.500573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.500951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:57.501474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:57.501474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.501474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:57.501474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.501474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:57.501474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.501474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.501474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:57.501474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.501474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:57.504469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:57.504913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:57.505104 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:57.505325 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:00:57.506431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:57.506678 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.507338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:57.507338 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.507338 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:57.507338 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.507338 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.507338 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:57.507338 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.507338 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:57.508900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.509392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:57.509392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.509392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:57.509392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.509392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.509392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:57.509392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.509392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.510517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:57.510858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.511204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.511204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:57.511204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.511204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.512106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:57.512591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.513098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.513098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:57.513098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.513098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:57.513098 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:57.513383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:57.513528 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:57.513688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:57.514028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:57.514312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.514817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:00:57.514817 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.514817 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:57.514817 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.514817 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.514817 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:57.514817 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.514817 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:00:57.516152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.516587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:00:57.516587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.516587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:00:57.516587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.516587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.516587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:00:57.516587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.516587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:57.517727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.518105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.518500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:57.518500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.518500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.519555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:57.520115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.520645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.520645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:57.520645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.520645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:00:57.521467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:00:57.521777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:00:57.521937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:00:57.522141 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:00:57.522433 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:57.522709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:00:57.522891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.523491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:00:57.523491 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.523491 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:00:57.523491 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.523491 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.523491 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:00:57.523491 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.523491 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.524488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.524981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:00:57.525563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.525563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:00:57.525855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:00:57.526349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:00:57.526854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:00:57.526854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:00:57.526854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:02.285165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:02.286304 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:02.286304 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:02.287446 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:02.318538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:02.318538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.319160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:02.319160 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.319160 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:02.319160 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.319160 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.319160 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:02.319160 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.319160 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.319929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.321499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:02.322203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:02.323649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.325681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:02.325681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.325681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.325681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:02.325681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.325681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:02.494974 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:02.495955 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:02.496320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:02.497575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:02.498190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:02.498465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.499673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:01:02.499673 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.499673 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:02.499673 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.499673 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.499673 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:02.499673 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.499673 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.501503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.502016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:02.502565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.502565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:02.503255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.503948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.504696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:02.504696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.504696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:02.505383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:02.505635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:02.506008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:02.506008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:02.506422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:02.506698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.507595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:02.507595 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.507595 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:02.507595 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.507595 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.507595 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:02.507595 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.507595 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:02.509017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.510449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:02.510449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.510449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:02.510449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.510449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.510449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:02.510449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.510449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.511106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:02.511698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.512401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.512401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:02.512401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.512401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.512572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:02.513276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.514077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.514077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:02.514077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.514077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:02.514077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:02.514077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:02.514376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:02.514376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:02.516794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:02.517286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.518160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:02.518160 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.518160 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:02.518160 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.518160 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.518160 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:02.518160 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.518160 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:02.519550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.520011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:02.520011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.520011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:02.520011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.520011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.520011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:02.520011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.520011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:02.520907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.521465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.521860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:02.521860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.521860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.523023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:02.523517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.524062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.524062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:02.524062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.524062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:02.524062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:02.524062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:02.524062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:02.524062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:02.524375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:02.524558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:02.524713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.525280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:02.525280 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.525280 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:02.525280 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.525280 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.525280 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:02.525280 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.525280 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.526101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:02.526539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.526984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:02.527123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:02.527674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:02.528232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:02.528232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:07.285892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:07.286972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:07.286972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:07.288181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:07.326016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:07.326290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.326917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:07.326917 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.326917 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:07.326917 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.326917 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.326917 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:07.326917 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.326917 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.327937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.328675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:07.329341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:07.330073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.331115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:07.331115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.331115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.331115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:07.331115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.331115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:07.495905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:07.496769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:07.497046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:07.498472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:07.498827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:07.499076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.500009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:01:07.500009 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.500009 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:07.500009 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.500009 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.500009 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:07.500009 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.500009 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.501717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.502404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:07.502956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.502956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:07.503943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.504724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.505610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:07.505610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.505610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:07.505610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:07.506423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:07.506662 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:07.506943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:07.507571 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:07.507915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.508767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 14:01:07.508767 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.508767 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:07.508767 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.508767 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.508767 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:07.508767 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.508767 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:07.510823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:07.511441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.511441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:07.511441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.511441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:07.511441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.511441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.511441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:07.511441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.511441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.512066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:07.512455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.512840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.512840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:07.512840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.512840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:07.513219 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.513663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:07.514227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.514227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:07.514227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.514227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.514227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:07.514227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.514227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:07.514227 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:07.514751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:07.514823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:07.514944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:07.515461 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:07.515830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.516566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:07.516566 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.516566 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:07.516566 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.516566 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.516566 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:07.516566 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.516566 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:07.517936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.518488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:07.518488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.518488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:07.518488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.518488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.518488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:07.518488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.518488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:07.519413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.519808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.520190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:07.520190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.520190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.520620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:07.521124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.521660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:07.523755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:07.524044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:07.524114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:07.524225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:07.524804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:07.525282 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:07.525614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.526565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:07.526565 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.526565 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:07.526565 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.526565 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.526565 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:07.526565 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.526565 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.527240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:07.527642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:07.527985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:07.528484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:07.529000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:07.529000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:07.529000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:12.286374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:12.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:12.287104 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:12.287869 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:12.335350 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:12.335609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.336627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:12.336627 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.336627 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:12.336627 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.336627 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.336627 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:12.336627 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.336627 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.337670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.338064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:12.338620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:12.339410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.341841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:12.341841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.341841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.341841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:12.341841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.341841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:12.496854 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:12.497512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:12.497715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:12.498466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:12.498663 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:12.498842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.499493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:01:12.499493 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.499493 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:12.499493 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.499493 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.499493 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:12.499493 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.499493 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.500517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.500863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:12.501178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.501178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:12.501691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.502108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.502588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:12.502588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.502588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:12.506250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:12.506959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:12.507245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:12.507514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:12.507938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:12.508133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.508689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Traceback (most recent call last):
Apr 21 14:01:12.508689 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.508689 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:12.508689 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.508689 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.508689 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:12.508689 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.508689 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:12.509565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:12.509970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.509970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:12.509970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.509970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:12.509970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.509970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.509970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:12.509970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.509970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.510530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:12.511196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.511996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.511996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:12.511996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.511996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:12.512194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.512597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:12.513037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.513037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:12.513037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.513037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.513037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:12.513037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.513037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:12.515024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:12.515742 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:12.515900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:12.516073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:12.516643 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:12.516998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.517787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:12.517787 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.517787 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:12.517787 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.517787 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.517787 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:12.517787 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.517787 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:12.519463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.519883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:12.519883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.519883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:12.519883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.519883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.519883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:12.519883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.519883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:12.521846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.522254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.522804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:12.522804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.522804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.524619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:12.525070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.525536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.525536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:12.525536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.525536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:12.525536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:12.526018 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:12.526018 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:12.526361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:12.526813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:12.527178 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:12.527444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.528192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:12.528192 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.528192 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:12.528192 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.528192 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.528192 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:12.528192 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.528192 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.528900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:12.529666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:12.530012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:12.530485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:12.530983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:12.530983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:12.530983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:17.287063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:17.287823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:17.287908 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:17.289290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:17.323380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:17.324074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.324326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:17.324326 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.324326 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:17.324326 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.324326 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.324326 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:17.324326 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.324326 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.326459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.327742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:17.330691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:17.331532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.332234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:17.332234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.332234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.332234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:17.332234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.332234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:17.497825 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:17.498427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:17.498698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:17.499989 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:17.500371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:17.500602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.501645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:01:17.501645 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.501645 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:17.501645 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.501645 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.501645 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:17.501645 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.501645 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.503601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.504111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:17.504658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.504658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:17.505303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.505758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.506307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:17.506307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.506307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:17.506796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:17.507479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:17.507608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:17.507774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:17.508342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:17.508674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.509520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:17.509520 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.509520 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:17.509520 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.509520 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.509520 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:17.509520 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.509520 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:17.510985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.511559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:17.511559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.511559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:17.511559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.511559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.511559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:17.511559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.511559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.512655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:17.513449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.513786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.513786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:17.513786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.513786 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.514787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:17.515183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.515807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.515807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:17.515807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.515807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:17.515807 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:17.516057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:17.516307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:17.516557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:17.517078 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:17.517476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.518157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:17.518157 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.518157 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:17.518157 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.518157 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.518157 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:17.518157 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.518157 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:17.519664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.520183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:17.520183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.520183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:17.520183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.520183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.520183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:17.520183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.520183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:17.521358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.521790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.522173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:17.522173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.522173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.522465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:17.522971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.523524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.523524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:17.523524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.523524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:17.525768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:17.526180 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:17.526403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:17.526653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:17.527015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:17.527405 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:17.527635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.528387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:01:17.528387 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.528387 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:17.528387 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.528387 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.528387 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:17.528387 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.528387 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.529799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.530167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:17.530542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.530542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:17.531513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:17.532146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:17.532782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:17.532782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:17.532782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:22.288276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:22.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:22.289525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:22.290833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:22.324484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:22.324766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.325523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:22.325523 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.325523 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:22.325523 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.325523 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.325523 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:22.325523 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.325523 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.326623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.327253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:22.327831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:22.328737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.329852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:22.329852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.329852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.329852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:22.329852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.329852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:22.498945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:22.499633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:22.499866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:22.501514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:22.501768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:22.501931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.502724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:01:22.502724 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.502724 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:22.502724 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.502724 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.502724 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:22.502724 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.502724 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.504235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.505122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:22.505905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.505905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:22.506256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.507105 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:22.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:22.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:22.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:22.508335 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:22.508818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:22.508990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:22.509200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.509809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:01:22.509809 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.509809 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:22.509809 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.509809 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.509809 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:22.509809 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.509809 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:22.511136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:22.511626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.511626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:22.511626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.511626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:22.511626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.511626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.511626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:22.511626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.511626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.512661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:22.513097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.513503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.513503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:22.513503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.513503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:22.514447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.514924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:22.515501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.515501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:22.515501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.515501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.515501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:22.515501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.515501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:22.516231 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:22.516684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:22.516816 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:22.516976 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:22.517585 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:22.517910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.518652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:22.518652 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.518652 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:22.518652 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.518652 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.518652 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:22.518652 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.518652 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:22.519660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.520597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:22.521285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:22.521957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:22.522599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.523526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:22.526376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:22.526825 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:22.526970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:22.527196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:22.527713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:22.528062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:22.528421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.529168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:22.529168 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.529168 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:22.529168 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.529168 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.529168 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:22.529168 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.529168 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.530716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:22.531129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.531483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:22.532562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:22.533041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:22.533567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:22.533567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:27.289249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:27.290106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:27.290191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:27.291820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:27.327162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:27.327484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.328138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:27.328138 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.328138 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:27.328138 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.328138 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.328138 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:27.328138 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.328138 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.328915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.329489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:27.330058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:27.330779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.331565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:27.331565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.331565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.331565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:27.331565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.331565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:27.500243 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:27.500774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:27.500879 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:27.501950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:27.502366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:27.502632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.503484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:01:27.503484 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.503484 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:27.503484 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.503484 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.503484 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:27.503484 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.503484 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.504656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.505395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:27.505913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:27.506727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.507498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:27.507498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.507498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.507498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:27.507498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.507498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:27.508623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:27.508960 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:27.509158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:27.509413 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:27.509919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:27.510129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.510745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:01:27.510745 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.510745 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:27.510745 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.510745 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.510745 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:27.510745 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.510745 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:27.512109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:27.513185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.513185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:27.513185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.513185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:27.513185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.513185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.513185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:27.513185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.513185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.513949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:27.514870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:27.515931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.516837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:27.517581 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:27.517904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:27.517904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:27.518262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:27.518623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:27.518982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.519898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Traceback (most recent call last):
Apr 21 14:01:27.519898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.519898 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:27.519898 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.519898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.519898 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:27.519898 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.519898 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:27.521321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:27.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:27.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:27.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:27.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.521994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:27.522759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.522759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:27.522759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:27.522759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:27.522759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.522759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:27.522759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:27.522759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:27.522759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:27.522759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.522759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:27.522985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.523857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.523857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:27.523857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.523857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.524880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:27.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.525572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.525572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:27.525572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.525572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:27.526930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:27.527385 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:27.527534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:27.527783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:27.528374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:27.528616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:27.528770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.529352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:27.529352 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.529352 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:27.529352 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.529352 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.529352 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:27.529352 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.529352 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.530726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:27.531135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.531518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:27.532947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:27.533608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:27.534098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:27.534098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:32.290153 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:32.290813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:32.290908 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:32.291803 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:32.321993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:32.322340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.323061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:32.323061 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.323061 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:32.323061 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.323061 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.323061 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:32.323061 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.323061 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.323842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.324374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:32.324831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:32.325420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.326243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:32.326243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.326243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.326243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:32.326243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.326243 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:32.501152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:32.501833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:32.501833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:32.502805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:32.503201 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:32.503582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.504234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:01:32.504234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.504234 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:32.504234 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.504234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.504234 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:32.504234 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.504234 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.505145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.505752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:32.506089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.506089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:32.506429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.506899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.507462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:32.507462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.507462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:32.508731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:32.509422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:32.509651 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:32.509870 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:32.510285 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:32.510464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.511041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:01:32.511041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.511041 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:32.511041 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.511041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.511041 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:32.511041 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.511041 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:32.512564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:32.513839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.513839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:32.513839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.513839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:32.513839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.513839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.513839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:32.513839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.513839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.514406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:32.515089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:32.515878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.516611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:32.517688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:32.518353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:32.518500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:32.518657 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:32.519047 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:32.519345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.519873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:32.519873 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.519873 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:32.519873 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.519873 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.519873 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:32.519873 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.519873 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:32.521197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:32.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:32.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:32.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:32.522655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.523006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.523413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:32.523413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.523413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.524230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:32.524712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.525232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.525232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:32.525232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.525232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:32.527693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:32.528611 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:32.528752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:32.528990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:32.529292 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:32.529598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:32.529659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.530543 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:32.530543 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.530543 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:32.530543 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.530543 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.530543 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:32.530543 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.530543 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.531930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:32.532323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.532673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:32.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:32.534457 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:32.535053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:32.535053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:37.291291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:37.292060 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:37.292060 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:37.293381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:37.336864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:37.337304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.338488 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:37.338488 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.338488 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:37.338488 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.338488 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.338488 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:37.338488 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.338488 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.339583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.340461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:37.341230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:37.342096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.343656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:37.343656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.343656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.343656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:37.343656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.343656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:37.502305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:37.502597 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:37.502784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:37.503733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:37.504193 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:37.504505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.505555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:01:37.505555 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.505555 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:37.505555 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.505555 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.505555 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:37.505555 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.505555 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.506374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:37.508123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:37.508981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.509943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:37.509943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.509943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.509943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:37.509943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.509943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:37.509943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:37.510287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:37.510620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:37.510777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:37.511372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:37.511629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.512365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:01:37.512365 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.512365 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:37.512365 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.512365 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.512365 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:37.512365 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.512365 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:37.514017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:37.514566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.514566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:37.514566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.514566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:37.514566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.514566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.514566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:37.514566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.514566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.515912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:37.516374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.516789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.516789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:37.516789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.516789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:37.516971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.517538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:37.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:37.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:37.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:37.518085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:37.518667 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:37.518813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:37.518958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:37.519334 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:37.519578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.520224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Traceback (most recent call last):
Apr 21 14:01:37.520224 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.520224 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:37.520224 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.520224 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.520224 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:37.520224 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.520224 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:37.521498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:37.521906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.521906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:37.521906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.521906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:37.521906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.521906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.521906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:37.521906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.521906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:37.522438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.522438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:37.522438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:37.522438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:37.522438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.522438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:37.522438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:37.522438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:37.522438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:37.522438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.522438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:37.522954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.523360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.523360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:37.523360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.523360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-9dbdbda7-502e-4c5a-8bf8-08191f4ac939 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.523933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:37.524917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.525526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.525526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:37.525526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.525526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:37.528708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:37.529272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:37.529376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:37.529555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:37.530024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:37.530407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:37.530654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.531499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:37.531499 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.531499 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:37.531499 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.531499 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.531499 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:37.531499 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.531499 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.532437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:37.532836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.533238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:37.533633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:37.534137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:37.534709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:37.534709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:42.291805 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:42.292776 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:42.292776 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:42.293656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:42.295504 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=239 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 14:01:42.327679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:42.328108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.329046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:42.329046 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.329046 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:42.329046 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.329046 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.329046 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:42.329046 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.329046 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.330598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.331059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:42.331436 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:42.331955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.332870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:42.332870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.332870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.332870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:42.332870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.332870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:42.502592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:42.503136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:42.503707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:42.504855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:42.505294 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:42.505545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.506595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:01:42.506595 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.506595 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:42.506595 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.506595 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.506595 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:42.506595 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.506595 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.508648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.509590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:42.510509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:42.511201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.511815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:42.511815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.511815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.511815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:42.511815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.511815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:42.511815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:42.512760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:42.512760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:42.512760 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:42.513041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:42.513418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:42.516368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.517126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.517614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.518009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.518424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.518893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:42.519481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.519481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:42.519481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.519481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.519481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:42.519481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.519481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:42.519481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:42.519481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:42.519762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:42.519853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:42.520260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:42.520487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.521082 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:01:42.521082 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.521082 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:42.521082 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.521082 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.521082 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:42.521082 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.521082 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:42.522582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:42.523022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.523022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:42.523022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.523022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:42.523022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.523022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.523022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:42.523022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.523022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:42.523655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.523655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:42.523655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:42.523655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:42.523655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.523655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:42.523655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:42.523655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:42.523655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:42.523655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.523655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:42.524030 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.524462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.524462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:42.524462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.524462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.524753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:42.525271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.526075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.526075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:42.526075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.526075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:42.529167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:42.529616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:42.529750 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:42.530018 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:42.530572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:42.530952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:42.531180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.531939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:42.531939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.531939 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:42.531939 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.531939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.531939 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:42.531939 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.531939 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.533454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:42.533947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.534751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:42.535119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:42.536130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:42.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:42.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:47.292437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:47.293300 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:47.293448 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:47.294886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:47.332438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:47.332693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.333406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:47.333406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.333406 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:47.333406 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.333406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.333406 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:47.333406 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.333406 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.334582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.335263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:47.335892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:47.336689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.337696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:47.337696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.337696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.337696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:47.337696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.337696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:47.503366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:47.503933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:47.504126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:47.505419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:47.505837 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:47.506085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:01:47.507011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.507011 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:47.507011 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.507011 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.507011 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:47.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.507011 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.508641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.509297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:47.510051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.510051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:47.510598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.511414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.512034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:47.512034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.512034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:47.512617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:47.513226 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:47.513399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:47.513602 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:47.514053 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:47.514267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.514824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:47.514824 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.514824 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:47.514824 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.514824 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.514824 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:47.514824 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.514824 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:47.516179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.516697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:47.516697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.516697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:47.516697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.516697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.516697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:47.516697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.516697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.517622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:47.518010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.518389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.518389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:47.518389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.518389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.519434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:47.520047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.520641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.520641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:47.520641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.520641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:47.520641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:47.520641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:47.520641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:47.520902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:47.521533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:47.521911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.522634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:01:47.522634 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.522634 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:47.522634 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.522634 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.522634 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:47.522634 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.522634 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:47.523609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:47.524092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.524092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:47.524092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.524092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:47.524092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.524092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.524092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:47.524092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.524092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:47.524520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.524520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:47.524520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:47.524520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:47.524520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.524520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:47.524520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:47.524520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:47.524520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:47.524520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.524520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:47.524951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.525495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.525495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:47.525495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.525495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.526698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:47.527231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.527827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.527827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:47.527827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.527827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:47.529603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:47.530257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:47.530484 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:47.530800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:47.531288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:47.531562 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:47.531773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.532447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Traceback (most recent call last):
Apr 21 14:01:47.532447 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.532447 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:47.532447 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.532447 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.532447 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:47.532447 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.532447 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.534116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.534690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:47.535041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.535041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a462d257-9439-43e6-b07d-0a84fafc26f0 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:47.535367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:47.535808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:47.536349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:47.536349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:47.536349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:52.293172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:52.294069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:52.294312 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:52.295835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:52.366160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:52.366622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.367968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:52.367968 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.367968 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:52.367968 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.367968 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.367968 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:52.367968 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.367968 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.074 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.369811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.370159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:52.370448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.370448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:52.371521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.372564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.372989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:52.372989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.372989 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:52.503893 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:52.504332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:52.504457 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:52.505555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:52.505841 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:52.506055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.506848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:52.506848 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.506848 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:52.506848 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.506848 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.506848 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:52.506848 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.506848 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.508498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.509084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:52.509526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.509526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.509947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.510584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:52.511427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.511427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:52.513136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:52.513456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:52.513523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:52.513631 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:52.514071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:52.514325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.515103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:52.515103 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.515103 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:52.515103 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.515103 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.515103 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:52.515103 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.515103 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:52.516908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.517740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:52.517740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.517740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:52.517740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.517740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.517740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:52.517740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.517740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.518533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:52.518966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.519478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.519478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:52.519478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.519478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.521359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:52.522060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.522474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.522474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:52.522474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.522474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:52.522474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:52.522797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:52.522877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:52.522974 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:52.523333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:52.523580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.524005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:01:52.524005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.524005 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:52.524005 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.524005 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.524005 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:52.524005 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.524005 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:52.524549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:52.524915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.524915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:52.524915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.524915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:52.524915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.524915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.524915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:52.524915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.524915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:52.525196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.525196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:52.525196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:52.525196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:52.525196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.525196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:52.525196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:52.525196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:52.525196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:52.525196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.525196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:52.525461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.525770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.525770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:52.525770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.525770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.526070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:52.526494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.526909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.526909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:52.526909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.526909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:52.530107 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:52.530673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:52.530749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:52.530906 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:52.531267 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:52.531523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:52.531665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.532939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:52.532939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.532939 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:52.532939 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.532939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.532939 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:52.532939 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.532939 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.533987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:52.534487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.534918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:52.534959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:52.535442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:52.536012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:52.536012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:57.293736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:57.294527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:57.294527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:57.295739 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:57.335235 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:57.335826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.336425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:01:57.336425 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.336425 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:57.336425 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.336425 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.336425 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:57.336425 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.336425 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.337359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.338649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:57.339618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:57.340434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.341974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:57.341974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.341974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.341974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:57.341974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.341974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:57.504614 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:57.504866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:57.504994 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:57.506119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:57.506451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:57.506723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.507680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:57.507680 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.507680 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:57.507680 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.507680 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.507680 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:57.507680 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.507680 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.509899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.511148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.511813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:57.512526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.513500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.513500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:57.513500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.513500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:57.513666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:57.514326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:57.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:57.514660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:01:57.515705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:57.515705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.516677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:57.516677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.516677 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:57.516677 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.516677 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.516677 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:57.516677 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.516677 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:57.518188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.519136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:57.519136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.519136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:57.519136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.519136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.519136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:57.519136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.519136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.520780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:57.522305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.522670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:57.523104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:57.523664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:57.524052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:57.524052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:57.524512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:57.524869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.525658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:01:57.525658 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.525658 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:57.525658 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.525658 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.525658 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:57.525658 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.525658 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:01:57.526877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:01:57.527313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.527313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:01:57.527313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.527313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:01:57.527313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.527313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.527313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:01:57.527313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.527313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:01:57.528343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.528343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:57.528343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:57.528343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:57.528343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.528343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:57.528343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:57.528343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:57.528343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:57.528343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.528343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:57.528707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.529070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.529070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:57.529070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.529070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.530072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:57.530579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.531068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.531068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:57.531068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.531068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:01:57.531068 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:01:57.531068 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:01:57.531068 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:01:57.531335 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:01:57.531827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:57.532175 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:01:57.532491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.533185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:01:57.533185 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.533185 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:01:57.533185 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.533185 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.533185 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:01:57.533185 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.533185 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.534295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:01:57.534722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.535042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:01:57.535487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:01:57.535993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:01:57.536571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:01:57.536571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:02.294371 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:02.295170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:02.295390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:02.296450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:02.335973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:02.336352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.337362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:02.337362 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.337362 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:02.337362 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.337362 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.337362 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:02.337362 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.337362 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.338076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.338725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:02.339264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:02.339923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.341665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:02.341665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.341665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.341665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:02.341665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.341665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:02.506917 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:02.506917 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:02.506917 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:02.507358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:02.507505 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:02.507765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.508725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:02.508725 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.508725 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:02.508725 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.508725 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.508725 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:02.508725 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.508725 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.510683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.511565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.513982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:02.515128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.516891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.516891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:02.516891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.516891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:02.516891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:02.517073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:02.517288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:02.517533 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:02.518400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:02.518400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.519057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:02.519057 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.519057 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:02.519057 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.519057 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.519057 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:02.519057 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.519057 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:02.519946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.520519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:02.520519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.520519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:02.520519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.520519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.520519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:02.520519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.520519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.520840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:02.521327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.521885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.521885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:02.521885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.521885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.522062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:02.522666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.523356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.523356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:02.523356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.523356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:02.523356 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:02.523952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:02.524085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:02.524172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:02.524523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:02.524740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.525193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:02:02.525193 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.525193 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:02.525193 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.525193 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.525193 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:02.525193 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.525193 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:02.526648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:02.526942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.526942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:02.526942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.526942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:02.526942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.526942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.526942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:02.526942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.526942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:02.527875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.527875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:02.527875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:02.527875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:02.527875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.527875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:02.527875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:02.527875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:02.527875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:02.527875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.527875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:02.528077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.528337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.528337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:02.528337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.528337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.529460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:02.529934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.530460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.530460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:02.530460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.530460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:02.531045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:02.531775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:02.531775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:02.531775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:02.532079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:02.532374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:02.532542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.533374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:02.533374 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.533374 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:02.533374 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.533374 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.533374 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:02.533374 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.533374 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.533984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:02.534699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:02.535404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:02.536149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:02.537037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:02.537037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:02.537037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:07.295023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:07.296000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:07.296000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:07.297330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:07.337763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:07.338286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.339469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:07.339469 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.339469 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:07.339469 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.339469 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.339469 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:07.339469 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.339469 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.340559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.341074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:07.341518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.341518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:07.342382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.343408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.344287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:07.344287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.344287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:07.505986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:07.506514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:07.506624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:07.508450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:07.508450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:07.508450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.509533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:02:07.509533 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.509533 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:07.509533 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.509533 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.509533 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:07.509533 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.509533 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.511150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.511463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:07.511774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.511774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:07.512858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.513181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.513539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:07.513539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.513539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:07.516871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:07.517525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:07.517643 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:07.517778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:07.518257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:07.518832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.519307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:07.519307 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.519307 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:07.519307 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.519307 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.519307 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:07.519307 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.519307 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:07.520495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.521268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:07.521268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.521268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:07.521268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.521268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.521268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:07.521268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.521268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.521670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:07.522420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.523094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.523094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:07.523094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.523094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.524255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:07.525081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.526032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.526032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:07.526032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.526032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:07.526032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:07.526032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:07.526032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:07.526032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:07.527341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:07.527341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.528180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:07.528180 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.528180 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:07.528180 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.528180 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.528180 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:07.528180 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.528180 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:07.529348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.530027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:07.530027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.530027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:07.530027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.530027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.530027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:07.530027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.530027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:07.530273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.530611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.531044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:07.531044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.531044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.531201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:07.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.532235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.532235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:07.532235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.532235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:07.532235 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:07.532443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:07.532735 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:07.533145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:07.534438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:07.534438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:07.534438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.534717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:02:07.534717 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.534717 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:07.534717 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.534717 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.534717 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:07.534717 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.534717 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.535987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.536348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:07.537143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:07.537603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:07.538093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:07.538093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:07.538093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:07.538093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:07.538093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:07.538093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:12.295788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:12.296854 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:12.297034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:12.298454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:12.340589 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:12.341009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.342017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:12.342017 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.342017 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:12.342017 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.342017 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.342017 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:12.342017 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.342017 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.342790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.343600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:12.344118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:12.344728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.345535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:12.345535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.345535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.345535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:12.345535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.345535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:12.506624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:12.507333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:12.507535 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:12.508596 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:12.509052 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:12.509413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.510467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:02:12.510467 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.510467 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:12.510467 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.510467 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.510467 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:12.510467 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.510467 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.511963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.512609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:12.513319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.513319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:12.513629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.514458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.515437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:12.515437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.515437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:12.517358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:12.518158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:12.518345 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:12.518517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:12.519054 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:12.519297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.519875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:12.519875 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.519875 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:12.519875 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.519875 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.519875 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:12.519875 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.519875 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:12.521089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.521628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:12.521628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.521628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:12.521628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.521628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.521628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:12.521628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.521628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.522047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:12.522374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.522703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.522703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:12.522703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.522703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.523358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:12.523828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.524349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.524349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:12.524349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.524349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:12.525529 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:12.526128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:12.526307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:12.526476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:12.526899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:12.527136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.527710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:12.527710 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.527710 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:12.527710 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.527710 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.527710 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:12.527710 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.527710 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:12.529166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.529587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:12.529587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.529587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:12.529587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.529587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.529587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:12.529587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.529587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:12.530707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.531032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.531401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:12.531401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.531401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.533182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:12.534253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:12.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:12.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:12.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:12.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:12.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:12.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:12.535655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:12.535655 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.535944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:12.535944 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.535944 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:12.535944 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.535944 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.535944 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:12.535944 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.535944 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.536798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:12.537993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:12.539336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:12.540727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:12.541227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:12.541227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:12.541227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:17.296830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:17.299804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:17.299804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:17.299804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:17.334497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:17.334863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.336323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:17.336323 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.336323 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:17.336323 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.336323 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.336323 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:17.336323 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.336323 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.336688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.337041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:17.337665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:17.338089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.338809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:17.338809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.338809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.338809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:17.338809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.338809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:17.507631 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:17.508346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:17.508526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:17.509543 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:17.509830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:17.510063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.510986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:02:17.510986 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.510986 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:17.510986 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.510986 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.510986 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:17.510986 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.510986 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.512715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.513151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:17.513555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.513555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:17.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.515886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.516480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:17.516480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.516480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:17.517937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:17.518637 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:17.518743 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:17.518877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:17.526102 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:17.526675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:17.526823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:17.526999 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:17.533920 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:17.542498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:17.542664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.543703 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:17.545011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:17.545322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:17.545742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:17.545742 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.545742 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:17.545742 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.545742 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.545742 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:17.545742 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.545742 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:17.546158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.546380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:17.547996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:17.548866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.549924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:17.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.551368 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:17.552023 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.552763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:17.553697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.024 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:17.554874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:17.555428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:17.555838 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.556264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.556813 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:17.557357 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.557357 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.557733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:17.558150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.558526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:17.559760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:17.560277 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:17.560792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:17.560792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:22.297537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:22.298383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:22.298383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:22.299713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:22.328130 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:22.328590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.329508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:22.329508 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.329508 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:22.329508 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.329508 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.329508 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:22.329508 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.329508 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.330317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.331286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:22.331746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:22.332181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.332762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:22.332762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.332762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.332762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:22.332762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.332762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:22.508454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:22.509346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:22.509518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:22.510521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:22.510837 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:22.511090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.512073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:22.512073 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.512073 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:22.512073 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.512073 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.512073 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:22.512073 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.512073 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.513563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.514167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.514967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:22.515704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.516767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.516767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:22.516767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.516767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:22.518603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:22.519240 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:22.519453 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:22.519694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:22.520048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:22.520254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.520859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:02:22.520859 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.520859 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:22.520859 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.520859 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.520859 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:22.520859 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.520859 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:22.522330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:22.522810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.522810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:22.522810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.522810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:22.522810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.522810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.522810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:22.522810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.522810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.523682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:22.524061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.524481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.524481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:22.524481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.524481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:22.525576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.526031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:22.526542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.526542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:22.526542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.526542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.526542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:22.526542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.526542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:22.526751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:22.527100 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:22.527279 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:22.527464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:22.528159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:22.528586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.529339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:22.529339 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.529339 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:22.529339 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.529339 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.529339 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:22.529339 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.529339 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:22.530630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.531098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:22.531098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.531098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:22.531098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.531098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.531098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:22.531098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.531098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:22.531984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.532433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.532798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:22.532798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.532798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.533721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:22.534175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.534665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.534665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:22.534665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.534665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:22.543645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:22.544002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:22.544158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:22.544483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:22.544961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:22.545359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:22.545857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.547059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:22.547059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.547059 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:22.547059 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.547059 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.547059 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:22.547059 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.547059 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.548591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:22.549526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:22.550361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:22.551316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:22.552049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:22.552049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:22.552049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:27.298147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:27.299199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:27.299199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:27.300612 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:27.336730 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:27.337168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.338132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:27.338132 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.338132 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:27.338132 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.338132 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.338132 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:27.338132 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.338132 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.339268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.339890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:27.340567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:27.341439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.342791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:27.342791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.342791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.342791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:27.342791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.342791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:27.509454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:27.511042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:27.511042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:27.511751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:27.512122 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:27.512414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:27.516465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.517381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:27.517758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.517758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.517758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:27.517758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.517758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.518246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.518739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:27.519278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.519278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:27.519640 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:27.520232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:27.520232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:27.520232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:27.520845 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:27.521078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:27.523866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.524470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.525133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.525512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:27.525753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.525753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.526986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:27.527361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.527728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.527728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:27.527728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.527728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:27.527728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:27.527930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:27.528290 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:27.528502 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:27.529073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:27.529554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.530561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:27.530561 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.530561 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:27.530561 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.530561 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.530561 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:27.530561 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.530561 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:27.531608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.531896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:27.531896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.531896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:27.531896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.531896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.531896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:27.531896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.531896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:27.532597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.532848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.533127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:27.533127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.533127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.533862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:27.534394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.534909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.534909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:27.534909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.534909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:27.544139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:27.544635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:27.544842 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:27.545138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:27.545501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:27.545777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:27.545938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.546527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:02:27.546527 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.546527 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:27.546527 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.546527 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.546527 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:27.546527 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.546527 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.548405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.549004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:27.549646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.549646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:27.550324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:27.551032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:27.551889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:27.551889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:27.551889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:32.298965 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:32.299789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:32.299789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:32.301031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:32.334418 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:32.334418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.335067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:32.335067 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.335067 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:32.335067 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.335067 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.335067 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:32.335067 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.335067 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.335849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.336476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:32.337091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:32.338066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.338911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:32.338911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.338911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.338911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:32.338911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.338911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:32.510365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:32.511035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:32.511379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:32.512754 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:32.513120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:32.513422 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.514461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:02:32.514461 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.514461 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:32.514461 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.514461 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.514461 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:32.514461 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.514461 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.515986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.516673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:32.517412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.517412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:32.517726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.518573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.519399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:32.519399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.519399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:32.520232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:32.520777 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:32.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:32.521232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:32.521598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:32.521795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.522418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:02:32.522418 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.522418 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:32.522418 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.522418 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.522418 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:32.522418 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.522418 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:32.523776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:32.524300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.524300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:32.524300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.524300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:32.524300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.524300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.524300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:32.524300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.524300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.525250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:32.525763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.526314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.526314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:32.526314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.526314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:32.527106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.527616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:32.528206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.528206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:32.528206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.528206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.528206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:32.528206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.528206 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:32.528206 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:32.528483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:32.528483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:32.528694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:32.529302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:32.529692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.530862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:32.530862 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.530862 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:32.530862 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.530862 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.530862 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:32.530862 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.530862 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:32.532220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.532663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:32.532663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.532663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:32.532663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.532663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.532663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:32.532663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.532663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:32.533614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.533983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.534555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:32.534555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.534555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.535412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:32.535964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.536616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.536616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:32.536616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.536616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:32.544520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:32.544886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:32.544946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:32.545080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:32.545590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:32.545964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:32.546200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.547041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:32.547041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.547041 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:32.547041 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.547041 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.547041 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:32.547041 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.547041 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.548544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:32.548951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.549357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:32.550427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:32.550958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:32.551507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:32.551507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:37.299801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:37.303635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:37.303635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:37.303635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:37.346199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:37.346575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.348267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:37.348267 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.348267 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:37.348267 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.348267 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.348267 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:37.348267 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.348267 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.350398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.351638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:37.352453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:37.353464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.354567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:37.354567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.354567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.354567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:37.354567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.354567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:37.511309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:37.512168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:37.512404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:37.513680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:37.514028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:37.514239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.515547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:02:37.515547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.515547 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:37.515547 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.515547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.515547 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:37.515547 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.515547 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.517400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.518017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:37.518975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.518975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:37.519100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.519860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.520688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:37.520688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.520688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:37.520917 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:37.521181 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:37.521400 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:37.521558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:37.521916 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:37.522187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.522828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:02:37.522828 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.522828 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:37.522828 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.522828 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.522828 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:37.522828 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.522828 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:37.523748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:37.524553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.524553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:37.524553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.524553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:37.524553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.524553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.524553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:37.524553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.524553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.524997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:37.525608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:37.526279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.526768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.527299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:37.528471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:37.528993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:37.529150 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:37.529390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:37.530035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:37.530495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.531356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:37.531356 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.531356 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:37.531356 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.531356 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.531356 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:37.531356 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.531356 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:37.532162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.532781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:37.532781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.532781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:37.532781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.532781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.532781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:37.532781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.532781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:37.533079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.533534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.534020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:37.534020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.534020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.534651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:37.535369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.535973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.535973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:37.535973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.535973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:37.545032 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:37.545660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:37.545860 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:37.546182 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:37.546584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:37.546820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:37.546963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.547622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:02:37.547622 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.547622 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:37.547622 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.547622 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.547622 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:37.547622 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.547622 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.549187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.549903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:37.550483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.550483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:37.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:37.551947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:37.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:37.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:37.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:42.300610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:42.301503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:42.301747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:42.303082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:42.305526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 14:02:42.340152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:42.340668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.341429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:42.341429 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.341429 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:42.341429 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.341429 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.341429 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:42.341429 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.341429 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.342455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.343343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:42.344041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:42.345065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.346646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:42.346646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.346646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.346646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:42.346646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.346646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:42.512126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:42.513188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:42.513467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:42.514916 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:42.515358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:42.515611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.516665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:02:42.516665 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.516665 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:42.516665 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.516665 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.516665 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:42.516665 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.516665 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.519060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.519644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:42.520270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.520270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:42.522005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.524953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.525800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:42.525800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.525800 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:42.525800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:42.525800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:42.525800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:42.525800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:42.525800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:42.526097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:42.529195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.529777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.530187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.530523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:42.530850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.530850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.531712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:42.532679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.533789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.533789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:42.533789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.533789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:42.533789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:42.533789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:42.533789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:42.533789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:42.534136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:42.534136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.535373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:42.535373 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.535373 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:42.535373 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.535373 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.535373 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:42.535373 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.535373 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:42.536387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.536692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:42.536692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.536692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:42.536692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.536692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.536692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:42.536692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.536692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:42.537295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.537570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.537812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:42.537812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.537812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.538593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:42.539057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.539393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.539393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:42.539393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.539393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:42.545668 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:42.546076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:42.546288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:42.546607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:42.547136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:42.547540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:42.547877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.548700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:02:42.548700 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.548700 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:42.548700 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.548700 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.548700 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:42.548700 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.548700 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.550294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.550605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:42.550931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.550931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:42.551904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:42.552345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:42.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:42.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:42.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:47.301638 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:47.302738 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:47.302837 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:47.304283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:47.339602 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:47.339969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.341265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:47.341265 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.341265 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:47.341265 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.341265 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.341265 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:47.341265 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.341265 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.342841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.343749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:47.344192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.344192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:47.344712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.345730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.346203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:47.346203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.346203 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:47.513008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:47.513559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:47.513775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:47.514851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:47.515196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:47.516423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.516423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:02:47.516423 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.516423 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:47.516423 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.516423 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.516423 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:47.516423 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.516423 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.518101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.518499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:47.518851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.518851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:47.520227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.520694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.521460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:47.521460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.521460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:47.522925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:47.523169 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:47.523281 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:47.523457 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:47.523959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:47.524319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.525086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:02:47.525086 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.525086 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:47.525086 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.525086 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.525086 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:47.525086 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.525086 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:47.525877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:47.526512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.526512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:47.526512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.526512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:47.526512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.526512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.526512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:47.526512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.526512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.526894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:47.527389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:47.527962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.528603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.529512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:47.532310 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:47.533116 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:47.533226 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:47.533431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:47.534066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:47.534450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.535176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:47.535176 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.535176 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:47.535176 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.535176 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.535176 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:47.535176 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.535176 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:47.536573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.537317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:47.537317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.537317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:47.537317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.537317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.537317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:47.537317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.537317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:47.537676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.538176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.538951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:47.538951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.538951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.541462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:47.542093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.542658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.542658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:47.542658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.542658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:47.546202 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:47.546648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:47.546841 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:47.547185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:47.547808 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:47.548304 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:47.548629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.549831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:47.549831 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.549831 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:47.549831 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.549831 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.549831 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:47.549831 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.549831 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.551858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:47.552418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.552857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:47.554221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:47.554807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:47.555319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:47.555319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:52.302363 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:52.303451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:52.303451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:52.304450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:52.337051 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:52.337262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.338001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:52.338001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.338001 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:52.338001 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.338001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.338001 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:52.338001 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.338001 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.338689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.339271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:52.339837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:52.340603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.341447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:52.341447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.341447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.341447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:52.341447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.341447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:52.513562 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:52.514206 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:52.514495 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:52.515553 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:52.515917 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:52.516232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.517086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:02:52.517086 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.517086 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:52.517086 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.517086 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.517086 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:52.517086 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.517086 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.518021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.518684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:52.519103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:52.519545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.520023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:52.520023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.520023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.520023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:52.520023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.520023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:52.523320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:52.523594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:52.523711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:52.523834 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:52.524384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:52.524685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.525344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:02:52.525344 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.525344 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:52.525344 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.525344 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.525344 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:52.525344 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.525344 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:52.526062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:52.526677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.526677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:52.526677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.526677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:52.526677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.526677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.526677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:52.526677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.526677 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.526982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:52.527397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.527776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.527776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:52.527776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.527776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:52.527904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.528404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:52.528851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.528851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:52.528851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.528851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.528851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:52.528851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.528851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:52.532840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:52.533287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:52.533437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:52.533674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:52.534617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:52.534911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.535621 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:52.535621 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.535621 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:52.535621 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.535621 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.535621 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:52.535621 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.535621 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:52.536602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.537132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:52.537132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.537132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:52.537132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.537132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.537132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:52.537132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.537132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:52.537737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.538298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.538891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:52.538891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.538891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.539048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:52.539834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.540690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.540690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:52.540690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.540690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:52.546705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:52.547253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:52.547398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:52.547616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:52.548288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:52.548751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:52.549034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.549839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:52.549839 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.549839 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:52.549839 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.549839 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.549839 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:52.549839 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.549839 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.550840 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:52.551261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.551574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:52.552052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:52.552549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:52.553001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:52.553001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:57.302947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:57.303722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:57.303905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:57.305107 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:57.340707 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:57.341103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.341814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:02:57.341814 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.341814 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:57.341814 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.341814 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.341814 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:57.341814 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.341814 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.342526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.343170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:57.343976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:57.344883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.346962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:57.346962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.346962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.346962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:57.346962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.346962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:57.514794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:57.515351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:57.515555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:57.516574 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:57.517000 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:57.517441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.518056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:02:57.518056 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.518056 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:57.518056 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.518056 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.518056 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:57.518056 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.518056 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.519053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.520026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:57.520721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:57.521114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.522509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:57.522509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.522509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.522509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:57.522509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.522509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:57.523690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:57.525013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:57.525241 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:57.525477 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:02:57.526127 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:57.526498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.527327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:02:57.527327 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.527327 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:57.527327 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.527327 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.527327 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:57.527327 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.527327 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:57.529033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:57.529574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.529574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:57.529574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.529574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:57.529574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.529574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.529574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:57.529574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.529574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.530937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:57.531813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:57.532732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.533656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:57.536648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.538330 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:02:57.539328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.540464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:57.541065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:57.541454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:57.541779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.542281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:02:57.547229 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:02:57.547919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:02:57.548203 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:02:57.548552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:02:57.548926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:57.549241 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:02:57.549379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.549913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:02:57.549913 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.549913 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:02:57.549913 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.549913 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.549913 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:02:57.549913 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.549913 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.551616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:02:57.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.553350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:02:57.553465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:02:57.554744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:02:57.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:02:57.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:02.303787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:02.304460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:02.304460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:02.305689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:02.338556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:02.339042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.339951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:02.339951 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.339951 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:02.339951 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.339951 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.339951 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:02.339951 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.339951 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.340735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.341318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:02.341881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:02.342668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.343610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:02.343610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.343610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.343610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:02.343610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.343610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:02.516398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:02.516398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:02.516398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:02.517466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:02.517735 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:02.517893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.518633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:03:02.518633 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.518633 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:02.518633 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.518633 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.518633 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:02.518633 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.518633 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.520197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.520797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:02.521348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.521348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:02.522355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.523108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.523850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:02.523850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.523850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:02.524877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:02.525231 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:02.525466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:02.525676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:02.526338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:02.526746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.527697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:02.527697 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.527697 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:02.527697 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.527697 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.527697 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:02.527697 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.527697 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:02.529091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:02.529519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.529519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:02.529519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.529519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:02.529519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.529519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.529519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:02.529519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.529519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.531046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:02.531410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.531812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.531812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:02.531812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.531812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:02.532918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.533442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:02.533891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.533891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:02.533891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.533891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.533891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:02.533891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.533891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:02.533891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:02.534121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:02.534332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:02.534503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:02.535104 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:02.535587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.536442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:02.536442 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.536442 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:02.536442 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.536442 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.536442 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:02.536442 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.536442 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:02.537809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.538288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:02.538288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.538288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:02.538288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.538288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.538288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:02.538288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.538288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:02.539392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.539718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.540100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:02.540100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.540100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.541349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:02.541830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.542379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.542379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:02.542379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.542379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:02.547679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:02.548257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:02.548431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:02.548662 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:02.549036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:02.549319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:02.549465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.550391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:02.550391 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.550391 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:02.550391 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.550391 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.550391 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:02.550391 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.550391 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.551882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:02.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.553122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:02.553567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:02.554604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:02.555432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:02.555432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:07.304370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:07.305318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:07.305318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:07.306954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:07.344288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.344881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.345190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:07.345558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.345959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:07.516232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:07.516623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:07.516722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:07.517834 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:07.518231 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:07.518600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.519557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:03:07.519557 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.519557 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:07.519557 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.519557 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.519557 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:07.519557 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.519557 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.521260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.522029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:07.522844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:07.523794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.525101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:07.525101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.525101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.525101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:07.525101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.525101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:07.525417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:07.525417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:07.525571 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:07.525749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:07.526204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:07.526549 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.527315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:07.527315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.527315 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:07.527315 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.527315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.527315 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:07.527315 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.527315 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:07.528025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:07.528564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.528564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:07.528564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.528564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:07.528564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.528564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.528564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:07.528564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.528564 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.528860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:07.529257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.529673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.529673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:07.529673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.529673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:07.530439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.530928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:07.531478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.531478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:07.531478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.531478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.531478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:07.531478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.531478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:07.534579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:07.534927 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:07.535058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:07.535225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:07.535750 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:07.536124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.536898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:07.536898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.536898 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:07.536898 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.536898 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.536898 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:07.536898 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.536898 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:07.538224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.538758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:07.538758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.538758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:07.538758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.538758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.538758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:07.538758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.538758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:07.539779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.540140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.540569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:07.540569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.540569 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.541691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:07.542183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.542723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.542723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:07.542723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.542723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:07.548323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:07.548813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:07.548972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:07.549224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:07.549612 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:07.549878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:07.550023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.550547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:07.550547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.550547 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:07.550547 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.550547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.550547 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:07.550547 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.550547 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.552034 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:07.552486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.552906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:07.553711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:07.554701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:07.555715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:07.555715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:12.305139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:12.306417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:12.306417 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:12.307421 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:12.342262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:12.342573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.343272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:12.343272 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.343272 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:12.343272 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.343272 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.343272 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:12.343272 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.343272 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.344201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.344929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:12.345649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:12.346663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.347627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:12.347627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.347627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.347627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:12.347627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.347627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:12.517107 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:12.517560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:12.517771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:12.518881 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:12.519243 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:12.519483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.520386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:03:12.520386 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.520386 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:12.520386 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.520386 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.520386 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:12.520386 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.520386 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.521856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.522360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:12.522717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.522717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:12.523618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.524084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.524632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:12.524632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.524632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:12.525501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:12.526043 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:12.526161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:12.526311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:12.526793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:12.527092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.527749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:12.527749 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.527749 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:12.527749 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.527749 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.527749 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:12.527749 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.527749 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:12.528599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:12.529593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.530144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.530532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:12.530893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.531519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:12.532138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.532138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:12.532138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.532138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.532138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:12.532138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.532138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:12.534937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:12.535190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:12.535266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:12.535431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:12.535781 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:12.536108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.536801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:12.536801 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.536801 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:12.536801 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.536801 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.536801 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:12.536801 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.536801 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:12.537976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.538456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:12.538456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.538456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:12.538456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.538456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.538456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:12.538456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.538456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:12.539360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.539899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.540443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:12.540443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.540443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.540975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:12.541766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.542315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.542315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:12.542315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.542315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:12.548897 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:12.549251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:12.549521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:12.549809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:12.550144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:12.550381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:12.550524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.551035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:12.551035 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.551035 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:12.551035 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.551035 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.551035 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:12.551035 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.551035 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.552503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:12.553380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:12.554528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:12.555434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:12.556186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:12.556186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:17.306011 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:17.306907 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:17.306907 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:17.307802 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:17.343857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:17.344322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.345301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:17.345301 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.345301 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:17.345301 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.345301 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.345301 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:17.345301 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.345301 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.346637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.347472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:17.348431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:17.349399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.350833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:17.350833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.350833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.350833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:17.350833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.350833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:17.517758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:17.518095 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:17.518389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:17.519343 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:17.519584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:17.519778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.520527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:03:17.520527 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.520527 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:17.520527 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.520527 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.520527 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:17.520527 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.520527 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.522186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.522854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:17.523470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.523470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:17.523616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.524447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.525287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:17.525287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.525287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:17.525918 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:17.526773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:17.527079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:17.527425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:17.527868 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:17.528113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.528849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:17.528849 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.528849 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:17.528849 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.528849 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.528849 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:17.528849 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.528849 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:17.530883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:17.531801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.531801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:17.531801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.531801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:17.531801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.531801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.531801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:17.531801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.531801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.532352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:17.532993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:17.533781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.534773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.535388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:17.535712 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:17.535956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:17.536168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:17.536427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:17.537012 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:17.537571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.539233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:17.539233 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.539233 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:17.539233 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.539233 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.539233 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:17.539233 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.539233 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:17.541055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.541524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:17.541524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.541524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:17.541524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.541524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.541524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:17.541524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.541524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:17.542653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.543022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.543404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:17.543404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.543404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.544068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:17.544574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.545073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.545073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:17.545073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.545073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:17.549345 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:17.549818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:17.549971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:17.550192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:17.550658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:17.550904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:17.551083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.551610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:17.551610 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.551610 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:17.551610 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.551610 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.551610 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:17.551610 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.551610 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.553147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:17.553737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:17.554895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:17.555606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:17.556378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:17.556378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:22.307113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:22.307823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:22.308054 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:22.310253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:22.351078 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:22.351558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.352857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:22.352857 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.352857 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:22.352857 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.352857 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.352857 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:22.352857 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.352857 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.354251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.355004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:22.355379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.355379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:22.355500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.356300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.356902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:22.356902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.356902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:22.518413 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:22.519067 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:22.520031 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:22.520392 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:22.520628 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:22.520799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.521563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:03:22.521563 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.521563 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:22.521563 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.521563 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.521563 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:22.521563 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.521563 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.523081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.523650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:22.524055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.524055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:22.524489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.525054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.525586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:22.525586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.525586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:22.526520 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:22.526789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:22.526902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:22.527020 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:22.527444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:22.527710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.528202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:22.528202 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.528202 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:22.528202 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.528202 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.528202 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:22.528202 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.528202 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:22.528964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:22.529497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.529497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:22.529497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.529497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:22.529497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.529497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.529497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:22.529497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.529497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.529781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:22.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.530565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.530565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:22.530565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.530565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:22.530794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.531274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:22.531781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.531781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:22.531781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.531781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.531781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:22.531781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.531781 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:22.535938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:22.536359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:22.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:22.536642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:22.537058 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:22.537509 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.538188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:22.538188 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.538188 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:22.538188 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.538188 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.538188 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:22.538188 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.538188 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.540152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:22.540152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.540152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:22.540152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.540152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.540152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:22.540152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.540152 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:22.541121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.541512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.541898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:22.541898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.541898 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.542933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:22.543456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.544227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.544227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:22.544227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.544227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:22.549898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:22.550512 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:22.550658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:22.550892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:22.551184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:22.551397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:22.551536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.551930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:22.551930 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.551930 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:22.551930 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.551930 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.551930 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:22.551930 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.551930 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.553502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:22.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.554799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:22.555170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:22.556011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:22.556888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:22.556888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:27.307709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:27.308588 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:27.308659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:27.309917 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:27.345549 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:27.345783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.346364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:27.346364 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.346364 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:27.346364 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.346364 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.346364 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:27.346364 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.346364 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.347085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.347631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:27.348331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:27.349043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.350161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:27.350161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.350161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.350161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:27.350161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.350161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:27.519307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:27.519835 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:27.520082 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:27.521344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:27.521709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:27.521946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.523627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:03:27.523627 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.523627 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:27.523627 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.523627 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.523627 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:27.523627 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.523627 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.524846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.525338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:27.525822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.525822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:27.526438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.527508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.529048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:27.529048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.529048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:27.529048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:27.529048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:27.529048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:27.529048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:27.529048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.530231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.530984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.531596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:27.532189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:27.533031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.533906 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.534662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:27.536991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:27.537938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:27.538122 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:27.538122 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:27.538473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:27.538707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.539072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:27.539072 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.539072 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:27.539072 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.539072 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.539072 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:27.539072 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.539072 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:27.539670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.539974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:27.539974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.539974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:27.539974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.539974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.539974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:27.539974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.539974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:27.540388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.540861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:27.541393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:27.541697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.542041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:27.542041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.542041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:27.542041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.542041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.542041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:27.542041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.542041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:27.550534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:27.551025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:27.551180 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:27.551475 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:27.551865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:27.552147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:27.552432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.553070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:27.553070 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.553070 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:27.553070 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.553070 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.553070 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:27.553070 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.553070 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.554577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:27.555568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.556296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:27.556847 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:27.558593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:27.559940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:27.559940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:32.308539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:32.309622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:32.309622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:32.310722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:32.349383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:32.349813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.350762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:32.350762 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.350762 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:32.350762 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.350762 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.350762 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:32.350762 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.350762 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.351590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.352149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:32.352548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:32.353004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.353560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:32.353560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.353560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.353560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:32.353560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.353560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:32.519880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:32.520334 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:32.520548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:32.521814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:32.522171 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:32.522497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.523432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:03:32.523432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.523432 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:32.523432 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.523432 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.523432 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:32.523432 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.523432 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.525018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.525625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:32.526137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.526137 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:32.527857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.528594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.529361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:32.529361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.529361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:32.529361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:32.529361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:32.529361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:32.529361 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:32.529953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:32.530572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.530969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:32.530969 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.530969 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:32.530969 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.530969 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.530969 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:32.530969 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.530969 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:32.532513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:32.533777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.533777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:32.533777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.533777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:32.533777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.533777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.533777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:32.533777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.533777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.534323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:32.534765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.535110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.535110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:32.535110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.535110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:32.535573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.536408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:32.537409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.537409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:32.537409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.537409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.537409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:32.537409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.537409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:32.537908 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:32.538148 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:32.538256 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:32.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:32.539124 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:32.539391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.539893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:32.539893 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.539893 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:32.539893 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.539893 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.539893 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:32.539893 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.539893 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:32.540798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.541244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:32.541244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.541244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:32.541244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.541244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.541244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:32.541244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.541244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:32.541716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.542412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:32.543151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:32.543571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:32.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:32.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:32.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:32.550972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:32.551505 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:32.551651 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:32.551888 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:32.552465 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:32.552862 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:32.553130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.554103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:32.554103 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.554103 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:32.554103 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.554103 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.554103 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:32.554103 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.554103 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.555807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:32.556170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.556513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:32.557414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:32.558091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:32.558803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:32.558803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:37.309266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:37.309827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:37.310004 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:37.311319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:37.345629 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:37.345799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.346204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:37.346204 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.346204 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:37.346204 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.346204 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.346204 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:37.346204 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.346204 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.346892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.347149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:37.347426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.347426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:37.347665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.347994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.348423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:37.348423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.348423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:37.520654 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:37.521183 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:37.521431 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:37.522527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:37.522716 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:37.522967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.524138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:03:37.524138 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.524138 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:37.524138 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.524138 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.524138 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:37.524138 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.524138 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.526375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:37.526957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.526957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:37.527481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.528976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.529794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:37.529794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.529794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:37.529794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:37.529794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:37.529794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:37.529794 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:37.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:37.530132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.530689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:37.530689 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.530689 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:37.530689 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.530689 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.530689 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:37.530689 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.530689 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:37.532016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:37.532471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.532471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:37.532471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.532471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:37.532471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.532471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.532471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:37.532471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.532471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.533995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:37.534380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.534777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.534777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:37.534777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.534777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:37.535726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.536269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:37.536734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.536734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:37.536734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.536734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.536734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:37.536734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.536734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:37.538420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:37.539185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:37.539437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:37.539651 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:37.540108 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:37.540389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.540910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:37.540910 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.540910 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:37.540910 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.540910 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.540910 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:37.540910 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.540910 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:37.542339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.542773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:37.542773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.542773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:37.542773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.542773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.542773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:37.542773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.542773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:37.543954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.544303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.544668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:37.544668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.544668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.545858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:37.546319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.546789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.546789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:37.546789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.546789 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:37.551564 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:37.551972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:37.552125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:37.552408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:37.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:37.553296 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:37.553538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.554298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:37.554298 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.554298 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:37.554298 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.554298 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.554298 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:37.554298 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.554298 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.555496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:37.555852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.556187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:37.556707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:37.557173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:37.558074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:37.558074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:42.309962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:42.311195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:42.311195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:42.312307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:42.314041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 14:03:42.345962 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:42.346288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.346812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:42.346812 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.346812 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:42.346812 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.346812 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.346812 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:42.346812 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.346812 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.347929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.348652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:42.349508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:42.350438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.351624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:42.351624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.351624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.351624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:42.351624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.351624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:42.521145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:42.521778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:42.521970 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:42.523247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:42.523631 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:42.523901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.524849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:03:42.524849 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.524849 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:42.524849 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.524849 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.524849 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:42.524849 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.524849 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.526410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.527027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:42.527633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.527633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:42.528248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.528969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.529616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:42.529616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.529616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:42.529616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:42.529740 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:42.529913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:42.530103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:42.530635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:42.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.531769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:42.531769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.531769 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:42.531769 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.531769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.531769 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:42.531769 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.531769 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:42.533040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:42.533536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.533536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:42.533536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.533536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:42.533536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.533536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.533536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:42.533536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.533536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.534372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:42.534867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.535265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.535265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:42.535265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.535265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:42.535785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.536251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:42.536765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.536765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:42.536765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.536765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.536765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:42.536765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.536765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:42.539025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:42.539341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:42.539430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:42.539565 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:42.539877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:42.540096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.540518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:42.540518 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.540518 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:42.540518 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.540518 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.540518 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:42.540518 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.540518 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:42.541234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.541639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:42.541639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.541639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:42.541639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.541639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.541639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:42.541639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.541639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:42.542088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.542451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.542942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:42.542942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.542942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.543180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:42.543652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.544103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.544103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:42.544103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.544103 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:42.552035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:42.552797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:42.552797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:42.552797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:42.553390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:42.553796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:42.554017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.556351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:42.556351 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.556351 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:42.556351 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.556351 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.556351 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:42.556351 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.556351 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.556848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:42.557600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:42.558237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:42.559107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:42.560480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:42.560480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:42.560480 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:47.310594 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:47.311379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:47.311541 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:47.312816 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:47.349502 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:47.349810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.350554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:47.350554 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.350554 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:47.350554 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.350554 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.350554 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:47.350554 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.350554 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.351332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.351758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:47.352151 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:47.352661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.353283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:47.353283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.353283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.353283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:47.353283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.353283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:47.522028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:47.523133 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:47.523317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:47.524771 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:47.525087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:47.525301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.526518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:03:47.526518 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.526518 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:47.526518 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.526518 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.526518 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:47.526518 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.526518 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.527811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.528233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:47.528631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.528631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:47.529019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.529756 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.530418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:47.530418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.530418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:47.530418 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:47.530688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:47.531013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:47.531330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:47.532049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:47.532481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.533406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:47.533406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.533406 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:47.533406 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.533406 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.533406 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:47.533406 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.533406 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:47.535347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:47.536302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.536302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:47.536302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.536302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:47.536302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.536302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.536302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:47.536302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.536302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.536791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:47.537333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:47.537793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.538468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.539310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:47.539767 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:47.539964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:47.540110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:47.540427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:47.541007 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:47.541492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.542317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:47.542317 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.542317 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:47.542317 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.542317 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.542317 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:47.542317 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.542317 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:47.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.544071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:47.544071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.544071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:47.544071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.544071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.544071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:47.544071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.544071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:47.545167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.545674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.546089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:47.546089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.546089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:47.546969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.547542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.547542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:47.547542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.547542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:47.552522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:47.553182 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:47.553458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:47.553534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:47.553985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:47.554382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:47.554601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.555315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:47.555315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.555315 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:47.555315 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.555315 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.555315 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:47.555315 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.555315 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.556833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:47.557306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.557706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:47.558408 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:47.558973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:47.559545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:47.559545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:52.311246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:52.311782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:52.311947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:52.312891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:52.348980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:52.349260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.350106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:52.350106 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.350106 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:52.350106 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.350106 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.350106 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:52.350106 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.350106 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.352265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.352670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:52.353090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.353090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:52.353593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.354027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.354568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:52.354568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.354568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:52.522915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:52.523249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:52.523372 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:52.524158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:52.524465 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:52.524703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.525506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Traceback (most recent call last):
Apr 21 14:03:52.525506 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.525506 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:52.525506 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.525506 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.525506 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:52.525506 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.525506 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.526431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.527033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-ae321cc2-6393-43ca-9172-3b594914c28d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:52.527606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:52.528533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.529388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:52.529388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.529388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.529388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:52.529388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.529388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:52.530307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:52.531041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:52.531294 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:52.531501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:52.532028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:52.532356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.533064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:52.533064 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.533064 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:52.533064 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.533064 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.533064 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:52.533064 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.533064 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:52.534598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:52.535173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.535173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:52.535173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.535173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:52.535173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.535173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.535173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:52.535173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.535173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.536064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:52.536490 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:52.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:52.537891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.538398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:52.538893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.538893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:52.538893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.538893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.538893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:52.538893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.538893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:52.540010 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:52.540230 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:52.540347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:52.540471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:52.540868 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:52.541086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.541629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:52.541629 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.541629 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:52.541629 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.541629 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.541629 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:52.541629 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.541629 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:52.542675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.543260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:52.543260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.543260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:52.543260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.543260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.543260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:52.543260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.543260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:52.543648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.544142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.544562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:52.544562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.544562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.545286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:52.545805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.546304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.546304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:52.546304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.546304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:52.553139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:52.553287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:52.553472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:52.553710 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:52.554192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:52.554715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:52.554715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.555049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:52.555049 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.555049 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:52.555049 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.555049 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.555049 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:52.555049 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.555049 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.556567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:52.557001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.557394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:52.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:52.559508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:52.560174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:52.560174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:57.311919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:57.312610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:57.312797 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:57.314121 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:57.351493 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:57.351859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.352494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:03:57.352494 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.352494 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:57.352494 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.352494 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.352494 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:57.352494 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.352494 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.353657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.354035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:57.354399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.354399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:57.354802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.355701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.356720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:57.356720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.356720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:57.523753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:57.524427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:57.524580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:57.525725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:57.526119 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:57.526429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.527390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:03:57.527390 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.527390 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:57.527390 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.527390 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.527390 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:57.527390 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.527390 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.529294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.529939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:57.530427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.530427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:57.531640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.532080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.532699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:57.532699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.532699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:57.532699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:57.532699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:57.532699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:57.532974 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:03:57.533364 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:57.533607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.534413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:57.534413 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.534413 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:57.534413 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.534413 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.534413 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:57.534413 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.534413 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:57.535699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.536193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:57.536193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.536193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:57.536193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.536193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.536193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:57.536193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.536193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.537022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:57.537750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.538531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.538531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:57.538531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.538531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.538907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:57.539916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.541367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.541367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:57.541367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.541367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:57.541367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:57.541367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:57.541367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:57.541367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:57.541929 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:57.541929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.542256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:57.542256 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.542256 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:57.542256 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.542256 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.542256 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:57.542256 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.542256 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:03:57.543592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:03:57.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:03:57.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:03:57.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.544050 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:57.545435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.545762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.546031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:57.546031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.546031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.547576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:57.547984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.548395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.548395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:57.548395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.548395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:03:57.553342 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:03:57.553722 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:03:57.553799 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:03:57.553979 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:03:57.554324 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:57.554554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:03:57.554710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.555340 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:03:57.555340 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.555340 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:03:57.555340 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.555340 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.555340 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:03:57.555340 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.555340 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.556779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:03:57.557460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:03:57.558001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:03:57.558779 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:03:57.559590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:03:57.559590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:03:57.559590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:02.312893 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:02.313814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:02.313814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:02.314791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:02.348151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:02.348631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:02.350739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:02.352667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.353887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:02.354771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:02.356346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.356346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:02.356346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.356346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:02.356346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.356346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.356346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:02.356346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.356346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:02.525264 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:02.525264 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:02.525627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:02.527190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:02.527190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:02.527190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.527914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:02.527914 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.527914 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:02.527914 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.527914 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.527914 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:02.527914 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.527914 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.529187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.529689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:02.530158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.530158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:02.530832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.531862 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.533403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:02.533403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.533403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:02.533403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:02.533403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:02.533403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:02.533403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:02.534998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:02.534998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.535111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:02.535111 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.535111 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:02.535111 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.535111 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.535111 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:02.535111 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.535111 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:02.536531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.537130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:02.537130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.537130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:02.537130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.537130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.537130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:02.537130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.537130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.537970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:02.538259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.538524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.538524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:02.538524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.538524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.540097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:02.540470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.540878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.540878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:02.540878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.540878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:02.541437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:02.541843 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:02.541995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:02.542173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:02.543035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:02.544267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.544371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:02.544371 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.544371 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:02.544371 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.544371 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.544371 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:02.544371 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.544371 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:02.545806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.546504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:02.546504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.546504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:02.546504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.546504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.546504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:02.546504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.546504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:02.547967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.548335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.548690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:02.548690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.548690 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.549192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:02.550405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.551559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.551559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:02.551559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.551559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:02.554009 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:02.554173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:02.554618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:02.554618 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:02.555072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:02.555748 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:02.555946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.556809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:02.556809 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.556809 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:02.556809 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.556809 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.556809 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:02.556809 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.556809 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:02.558641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.559054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:02.559639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:02.561107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:02.562024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:02.562024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:07.313961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:07.317360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:07.317360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:07.317360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:07.351539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:07.351880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.353327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:07.353327 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.353327 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:07.353327 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.353327 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.353327 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:07.353327 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.353327 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.353702 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.353982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:07.354314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.354314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:07.355175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.358027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.359123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:07.359123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.359123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:07.525285 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:07.525815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:07.526151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:07.527010 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:07.527341 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:07.527580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.528562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:07.528562 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.528562 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:07.528562 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.528562 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.528562 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:07.528562 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.528562 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.530095 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.531066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:07.531784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.531784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:07.531935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.533147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.536066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:07.536066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.536066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:07.536066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:07.536066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:07.536066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:07.536066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:07.536066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:07.536668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.536668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:07.536668 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.536668 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:07.536668 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.536668 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.536668 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:07.536668 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.536668 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:07.537150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.538172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:07.538172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.538172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:07.538172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.538172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.538172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:07.538172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.538172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.538627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:07.539314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.540294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.540294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:07.540294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.540294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.540976 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.541998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.541998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:07.541998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.541998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:07.541998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:07.542340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:07.542498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:07.542670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:07.543098 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:07.543390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.543934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:07.543934 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.543934 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:07.543934 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.543934 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.543934 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:07.543934 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.543934 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:07.545463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.545816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:07.545816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.545816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:07.545816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.545816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.545816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:07.545816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.545816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:07.546962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.547270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.547552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:07.547552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.547552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.548858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:07.549445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.550176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.550176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:07.550176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.550176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:07.554160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:07.554526 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:07.554684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:07.555003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:07.555546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:07.556036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:07.556403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.557330 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:07.557330 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.557330 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:07.557330 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.557330 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.557330 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:07.557330 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.557330 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.558706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:07.559365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.560003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:07.560343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:07.561199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:07.561719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:07.561719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:12.315513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:12.316251 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:12.316521 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:12.317930 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:12.353347 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:12.353946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.355036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:12.355036 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.355036 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:12.355036 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.355036 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.355036 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:12.355036 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.355036 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.355827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.356491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:12.357139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:12.357942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.358833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:12.358833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.358833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.358833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:12.358833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.358833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:12.525984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:12.526575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:12.526827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:12.527972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:12.528359 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:12.528622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.529663 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:12.529663 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.529663 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:12.529663 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.529663 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.529663 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:12.529663 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.529663 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.530973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.531830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:12.532708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:12.533783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.534921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:12.534921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.534921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.534921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:12.534921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.534921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:12.534921 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:12.534921 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:12.534921 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:12.534921 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:12.535413 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:12.535413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.535413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:12.535413 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.535413 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:12.535413 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.535413 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.535413 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:12.535413 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.535413 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:12.536125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.536907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:12.537625 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.537995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:12.538435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.538994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:12.542853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:12.543366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:12.543864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:12.543864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:12.544282 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:12.544628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.545252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:12.545252 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.545252 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:12.545252 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.545252 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.545252 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:12.545252 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.545252 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:12.546825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.547394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:12.547394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.547394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:12.547394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.547394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.547394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:12.547394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.547394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:12.548619 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.549159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.549687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:12.549687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.549687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.550386 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:12.550953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.551626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.551626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:12.551626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.551626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:12.554668 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:12.554870 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:12.554938 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:12.555039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:12.555299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:12.555524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:12.555699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.556176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:12.556176 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.556176 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:12.556176 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.556176 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.556176 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:12.556176 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.556176 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.556824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:12.557412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:12.557947 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:12.558654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:12.559309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:12.559309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:12.559309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:17.316239 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:17.316901 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:17.316961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:17.318864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:17.354774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:17.355044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.355868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:17.355868 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.355868 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:17.355868 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.355868 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.355868 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:17.355868 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.355868 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.356573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.357184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:17.357845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:17.358794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.359754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:17.359754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.359754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.359754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:17.359754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.359754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:17.526851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:17.527505 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:17.528132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:17.529245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:17.529663 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:17.529946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.530961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:17.530961 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.530961 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:17.530961 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.530961 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.530961 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:17.530961 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.530961 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.532817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.533447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:17.533877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.533877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:17.534894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.535881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.536485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:17.536485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.536485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:17.536485 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:17.536485 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:17.536485 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:17.536485 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:17.536795 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:17.537072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.537692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:17.537692 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.537692 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:17.537692 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.537692 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.537692 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:17.537692 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.537692 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:17.539178 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.539735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:17.539735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.539735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:17.539735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.539735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.539735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:17.539735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.539735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.540872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:17.541259 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.541695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.541695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:17.541695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.541695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.544493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:17.545020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.545635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.545635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:17.545635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.545635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:17.546054 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:17.546539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:17.546684 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:17.546876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:17.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:17.547618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.548247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:17.548247 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.548247 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:17.548247 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.548247 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.548247 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:17.548247 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.548247 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:17.549914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.550458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:17.550458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.550458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:17.550458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.550458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.550458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:17.550458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.550458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:17.552174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.552566 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.552941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:17.552941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.552941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.555503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:17.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.556537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.556537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:17.556537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.556537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:17.556906 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:17.557107 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:17.557186 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:17.557349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:17.557693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:17.557958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:17.558146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.558634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:17.558634 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.558634 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:17.558634 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.558634 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.558634 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:17.558634 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.558634 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.559572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:17.559984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.560328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:17.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:17.561675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:17.562164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:17.562164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:22.318184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:22.318184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:22.318184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:22.318184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:22.355680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:22.356051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.357071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:22.357071 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.357071 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:22.357071 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.357071 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.357071 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:22.357071 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.357071 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.358084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.358876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:22.359544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:22.360706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.361668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:22.361668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.361668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.361668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:22.361668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.361668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:22.527608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:22.528497 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:22.528730 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:22.530073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:22.530443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:22.530692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.532234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:22.532234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.532234 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:22.532234 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.532234 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.532234 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:22.532234 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.532234 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.534049 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.534459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:22.534812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.534812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:22.536432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.536921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.537440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:22.537440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.537440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:22.537440 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:22.538174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:22.538269 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:22.538387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:22.539354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:22.539681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.540633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:22.540633 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.540633 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:22.540633 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.540633 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.540633 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:22.540633 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.540633 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:22.541610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.542155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:22.542155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.542155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:22.542155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.542155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.542155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:22.542155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.542155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.542492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:22.542914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.543313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.543313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:22.543313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.543313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.543534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:22.544018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.544597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.544597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:22.544597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.544597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:22.546394 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:22.546881 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:22.547057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:22.547259 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:22.547709 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:22.547923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.548450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:22.548450 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.548450 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:22.548450 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.548450 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.548450 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:22.548450 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.548450 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:22.549882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.550830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:22.550830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.550830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:22.550830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.550830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.550830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:22.550830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.550830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:22.551955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.552661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.553068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:22.553068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.553068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.553795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:22.554323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.554911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.554911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:22.554911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.554911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:22.557203 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:22.557752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:22.557752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:22.558046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:22.558503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:22.558861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:22.559130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.559990 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:22.559990 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.559990 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:22.559990 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.559990 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.559990 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:22.559990 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.559990 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.561642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:22.562140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.562522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:22.562601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:22.563166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:22.563773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:22.563773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:27.317471 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:27.318097 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:27.318514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:27.319288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:27.355944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:27.356246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.357194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:27.357194 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.357194 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:27.357194 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.357194 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.357194 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:27.357194 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.357194 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.358742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.359443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:27.360039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.360039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:27.360404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.361180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.362081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:27.362081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.362081 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:27.528364 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:27.529086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:27.529419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:27.530679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:27.531013 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:27.531356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.532397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:27.532397 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.532397 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:27.532397 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.532397 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.532397 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:27.532397 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.532397 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.534399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.535089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:27.535736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.535736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:27.536393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.536913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.537443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:27.537443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.537443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:27.538164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:27.538576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:27.538719 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:27.538872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:27.539627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:27.539845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.540700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:27.540700 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.540700 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:27.540700 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.540700 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.540700 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:27.540700 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.540700 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:27.542230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.542730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:27.542730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.542730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:27.542730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.542730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.542730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:27.542730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.542730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.543747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:27.544807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.545227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:27.545704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.546298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:27.546298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:27.546298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.546298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:27.546298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.546298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:27.546298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.546298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.546298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:27.546298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.546298 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:27.546929 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:27.547537 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:27.547738 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:27.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:27.548349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:27.548572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.549167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:27.549167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.549167 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:27.549167 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.549167 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.549167 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:27.549167 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.549167 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:27.550470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.550933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:27.550933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.550933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:27.550933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.550933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.550933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:27.550933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.550933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:27.552101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.552499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.552872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:27.552872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.552872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.553965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:27.554487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.554992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.554992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:27.554992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.554992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:27.557630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:27.558014 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:27.558164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:27.558697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:27.559443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:27.559753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:27.559912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:27.560393 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.560393 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:27.560393 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.560393 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.560393 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:27.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.561230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:27.561576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.561903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:27.562158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:27.562639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:27.563128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:27.563128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:32.318180 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:32.319312 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:32.319312 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:32.320302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:32.402261 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:32.402732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.403855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:32.403855 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.403855 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:32.403855 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.403855 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.403855 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:32.403855 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.403855 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.085 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.404656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.405462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:32.406076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:32.406814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.407811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:32.407811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.407811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.407811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:32.407811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.407811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:32.529115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:32.529711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:32.529946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:32.531301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:32.531649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:32.531960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.532975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:32.532975 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.532975 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:32.532975 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.532975 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.532975 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:32.532975 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.532975 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.534654 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.535021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:32.535526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.535526 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:32.536744 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.537254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.537768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:32.537768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.537768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:32.538563 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:32.538961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:32.539043 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:32.539163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:32.539559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:32.539807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.540349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:32.540349 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.540349 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:32.540349 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.540349 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.540349 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:32.540349 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.540349 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:32.541233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.541706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:32.541706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.541706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:32.541706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.541706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.541706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:32.541706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.541706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.542264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:32.543128 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.543640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:32.544087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.545079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:32.545079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:32.545079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.545079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:32.545079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.545079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:32.545079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.545079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.545079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:32.545079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.545079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:32.547335 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:32.547820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:32.547998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:32.548147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:32.548716 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:32.549169 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.549917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:32.549917 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.549917 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:32.549917 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.549917 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.549917 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:32.549917 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.549917 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:32.551459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.552027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:32.552027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.552027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:32.552027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.552027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.552027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:32.552027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.552027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:32.552910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.553316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.553758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:32.553758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.553758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.554653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:32.555311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.555868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.555868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:32.555868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.555868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:32.558140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:32.558552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:32.558702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:32.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:32.559474 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:32.559916 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:32.560172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.560904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:32.560904 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.560904 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:32.560904 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.560904 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.560904 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:32.560904 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.560904 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.562607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:32.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.563447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:32.564527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:32.565451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:32.566267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:32.566267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:37.318966 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:37.319630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:37.319630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:37.321948 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:37.353934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:37.354170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.354844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:37.354844 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.354844 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:37.354844 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.354844 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.354844 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:37.354844 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.354844 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.355633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.356189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:37.356687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:37.357410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.358079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:37.358079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.358079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.358079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:37.358079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.358079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:37.530039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:37.530671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:37.530946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:37.531924 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:37.532172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:37.532354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.534531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:37.534531 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.534531 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:37.534531 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.534531 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.534531 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:37.534531 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.534531 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.535576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.536190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:37.536901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:37.537746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.538808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:37.538808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.538808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.538808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:37.538808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.538808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:37.539147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:37.539399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:37.539399 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:37.539633 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:37.540062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:37.540541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.541514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:37.541514 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.541514 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:37.541514 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.541514 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.541514 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:37.541514 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.541514 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:37.542306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.542761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:37.542761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.542761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:37.542761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.542761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.542761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:37.542761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.542761 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.543087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:37.543449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.543806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.543806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:37.543806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.543806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.544075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:37.544565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.545090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.545090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:37.545090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.545090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:37.547726 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:37.548283 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:37.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:37.548672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:37.549363 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:37.549835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.550742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:37.550742 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.550742 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:37.550742 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.550742 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.550742 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:37.550742 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.550742 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:37.551949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.552412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:37.552412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.552412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:37.552412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.552412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.552412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:37.552412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.552412 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:37.552869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.553196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.553590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:37.553590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.553590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.554166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:37.554628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.555085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.555085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:37.555085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.555085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:37.558685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:37.559006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:37.559101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:37.559340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:37.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:37.559997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:37.560190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.560820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:37.560820 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.560820 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:37.560820 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.560820 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.560820 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:37.560820 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.560820 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.561806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:37.562377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.562916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:37.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:37.563738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:37.564240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:37.564240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:42.320187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:42.321110 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:42.321426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:42.322856 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:42.325190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 14:04:42.358021 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:42.358021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.359102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:42.359102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.359102 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:42.359102 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.359102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.359102 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:42.359102 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.359102 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.360301 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.360793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:42.361270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.361270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:42.361487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.362014 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.362675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:42.362675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.362675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:42.530898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:42.531216 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:42.531581 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:42.533250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:42.533651 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:42.533908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:42.535092 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.535092 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:42.535092 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.535092 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.535092 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:42.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.535092 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.536391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.537248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:42.541313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:42.541857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.542403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:42.542403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.542403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.542403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:42.542403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.542403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:42.542403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:42.542403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:42.542403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:42.542403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.544808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.545664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:42.546404 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:42.547036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:42.547626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:42.548473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:42.549672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:42.550365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:42.550365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:42.550365 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:42.550365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.551065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:42.551065 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.551065 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:42.551065 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.551065 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.551065 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:42.551065 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.551065 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:42.553183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.553920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:42.553920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.553920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:42.553920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.553920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.553920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:42.553920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.553920 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:42.554468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.554866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.555269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:42.555269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.555269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.555590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:42.556114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.556673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.556673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:42.556673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.556673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:42.559299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:42.559956 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:42.560129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:42.560439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:42.560968 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:42.561638 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:42.561907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.562593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:42.562593 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.562593 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:42.562593 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.562593 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.562593 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:42.562593 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.562593 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.563486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:42.563880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.564272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:42.564580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:42.565051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:42.565642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:42.565642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:46.465811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:46.467784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:46.468125 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:46.483345 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 21 14:04:46.483552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106267) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 21 14:04:46.483552 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Fixing resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:04:46.519044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, resource={'id': 'b57c058f-054e-4a2a-980f-6bfda2111e90', 'name': 'private', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['6a3993e6-b5ba-4812-baa3-b25db5880cd9', 'c97deb27-a1d6-4e03-b489-95abde8fd791'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:29:32Z', 'updated_at': '2026-04-21T13:29:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 24380}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.519540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.520087 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3f6c78fa-b671-412d-bb34-5426a40d8bff, external_ids_update=None, columns={'tag': 24380}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.520320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.521374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:46.521374 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.521374 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:46.521374 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.521374 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.521374 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:46.521374 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.521374 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:46.522711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Failed to fix resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.522711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:46.522711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:04:46.522711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:04:46.522711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:04:46.522711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:04:46.522711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 14:04:46.522711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:46.522711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.522711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.522711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.523349 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.524033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:46.524033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.524033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:46.524033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Fixing resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:04:46.562429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d0607710-681d-444d-be26-dbeca5a07b46, resource={'id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'name': 'public', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-21T13:29:38Z', 'updated_at': '2026-04-21T13:29:48Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.562783 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d0607710-681d-444d-be26-dbeca5a07b46, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.563785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-19f04b40-ed99-482a-8774-a76892a716d1, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.564463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.565329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:46.565329 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.565329 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:46.565329 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.565329 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.565329 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:46.565329 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.565329 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:46.566778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Failed to fix resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.566778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:46.566778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:04:46.566778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:04:46.566778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:04:46.566778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:04:46.566778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 14:04:46.566778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:46.566778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.566778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.566778 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.567806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:46.567806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.567806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:46.567806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Fixing resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:04:46.600101 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, resource={'id': '345e31fe-ede8-4dd0-823e-e81dcbc5bef0', 'name': 'admin_net', 'tenant_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'project_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['bb271c4b-4172-418f-995c-f72abd439dbe'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:34:46Z', 'updated_at': '2026-04-21T13:34:47Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 22486}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.600796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.601420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-c1b4e94e-49ea-4c1f-8edf-2f1433fdff16, external_ids_update=None, columns={'tag': 22486}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.601616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.603473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:46.603473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.603473 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:46.603473 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.603473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.603473 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:46.603473 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.603473 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:46.605015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Failed to fix resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.605015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:46.605015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:04:46.605015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:04:46.605015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:04:46.605015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:04:46.605015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 14:04:46.605015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:46.605015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.605015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.605015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.605664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.606623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:46.606623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.606623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:46.606623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Fixing resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:04:46.639713 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, resource={'id': 'b364ad09-f70d-45d8-b9dd-8420f5f1c377', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['620e9559-2c15-4370-904f-f4677bc69542'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:35:24Z', 'updated_at': '2026-04-21T13:35:25Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 61006}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.640132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.640815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a100d4e8-7f11-40c9-8e4a-346c2c764f69, external_ids_update=None, columns={'tag': 61006}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.640971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.641995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:46.641995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.641995 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:46.641995 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.641995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.641995 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:46.641995 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.641995 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:46.643440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Failed to fix resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.643440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:46.643440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:04:46.643440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:04:46.643440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:04:46.643440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:04:46.643440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 14:04:46.643440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:46.643440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.643440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.643440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.645622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.646028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:46.646028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.646028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:46.646028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Fixing resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:04:46.734187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, resource={'id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'enable_snat': True}, 'gw_port_id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-61m1f', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'ip_address': '172.24.5.49', 'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'}, {'ip_address': '2001:db8::162', 'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 3, 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04'}, resource_type=routers, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.734984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.735391 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.735607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.735785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.736944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:46.736944 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.736944 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:46.736944 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.736944 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.736944 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:46.736944 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.736944 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:46.737701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Unable to update router 06899d4c-a436-45e4-bed5-95dabe888f51. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.738975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Failed to fix resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.738975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:46.738975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:04:46.738975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:04:46.738975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:04:46.738975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:04:46.738975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 21 14:04:46.738975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 21 14:04:46.738975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 21 14:04:46.738975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 21 14:04:46.738975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:46.739428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:46.739879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:46.739879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:46.739879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.739879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:46.739879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.739879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.739879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:46.739879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.739879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:46.739879 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Fixing resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:04:46.859836 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, resource={'id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'name': '', 'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'project_id': '', 'mac_address': 'fa:16:3e:fb:71:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000168688', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.860250 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3 ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'neutron:network_name': 'neutron-d0607710-681d-444d-be26-dbeca5a07b46', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-06899d4c-a436-45e4-bed5-95dabe888f51'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.49/24', '2001:db8::162/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.860672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, lrouter_port=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:46.860831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.862279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Traceback (most recent call last):
Apr 21 14:04:46.862279 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.862279 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:46.862279 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.862279 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.862279 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:46.862279 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.862279 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:46.863281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] Maintenance task: Failed to fix resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.863281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:46.863281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:04:46.863281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:04:46.863281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:04:46.863281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:04:46.863281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 21 14:04:46.863281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:46.863281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.863281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.863281 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:46.863937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:46.864766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:46.864766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:46.864766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:46.864766 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49fdca9b-5b95-4451-a11b-29d967233e63 None None] OVN maintenance task check_for_inconsistencies finished in 0.395 seconds
Apr 21 14:04:47.321040 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:47.321792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:47.321975 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:47.322937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:47.359577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:47.359894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.360950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:47.360950 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.360950 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:47.360950 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.360950 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.360950 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:47.360950 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.360950 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.362446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.363454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:47.364421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:47.365836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.367135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:47.367135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.367135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.367135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:47.367135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.367135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:47.532882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:47.532882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:47.532882 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:47.533911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:47.534416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:47.534694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.535769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:47.535769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.535769 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:47.535769 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.535769 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.535769 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:47.535769 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.535769 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.537478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.537855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:47.538210 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.538210 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.539171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.539696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:47.540271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.540271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:47.540649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:47.541001 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:47.541084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:47.541199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:47.541752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:47.542101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.542849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:47.542849 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.542849 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:47.542849 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.542849 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.542849 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:47.542849 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.542849 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:47.543606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.544100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:47.544100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.544100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:47.544100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.544100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.544100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:47.544100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.544100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.544429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:47.544791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.545199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.545199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:47.545199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.545199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.545727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:47.546347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.547066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.547066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:47.547066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.547066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:47.548695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:47.549003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:47.549277 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:47.549476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:47.549967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:47.550439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.550834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:47.550834 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.550834 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:47.550834 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.550834 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.550834 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:47.550834 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.550834 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:47.551605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.552080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:47.552080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.552080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:47.552080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.552080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.552080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:47.552080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.552080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:47.552583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.553022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.553522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:47.553522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.553522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.554630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:47.555414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.555937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.555937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:47.555937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.555937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:47.559847 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:47.560201 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:47.560333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:47.560492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:47.561351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:47.561801 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:47.562072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.562971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:47.562971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.562971 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:47.562971 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.562971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.562971 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:47.562971 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.562971 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.563955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:47.564337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.564668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:47.565118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:47.565674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:47.566162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:47.566162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:52.321959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:52.323682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:52.323682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:52.323682 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:52.357934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:52.358417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.359331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:52.359331 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.359331 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:52.359331 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.359331 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.359331 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:52.359331 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.359331 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.360362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.360977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:52.361601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:52.362291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.363182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:52.363182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.363182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.363182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:52.363182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.363182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:52.532924 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:52.534223 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:52.534557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:52.535878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:52.536279 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:52.536547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.537738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:52.537738 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.537738 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:52.537738 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.537738 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.537738 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:52.537738 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.537738 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.539693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.540391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:52.541066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.541066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.542397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.542890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:52.543460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.543460 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:52.543460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:52.543741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:52.543741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:52.543831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:52.544170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:52.544391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.545293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:52.545293 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.545293 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:52.545293 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.545293 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.545293 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:52.545293 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.545293 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:52.546391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.546890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:52.546890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.546890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:52.546890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.546890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.546890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:52.546890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.546890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.547296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:52.547931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.548622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:52.549432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:52.549973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:52.550454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:52.550454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:52.550454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.550681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:52.550681 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.550681 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:52.550681 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.550681 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.550681 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:52.550681 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.550681 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:52.551378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.551860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:52.551860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.551860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:52.551860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.551860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.551860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:52.551860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.551860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:52.552356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.553016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.553417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:52.553417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.553417 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.553504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:52.554535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.555640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.555640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:52.555640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.555640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:52.560199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:52.560592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:52.560724 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:52.560990 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:52.561694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:52.561913 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:52.562055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.562618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:52.562618 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.562618 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:52.562618 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.562618 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.562618 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:52.562618 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.562618 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.563901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:52.564441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.564672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:52.565767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:52.566573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:52.567087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:52.567087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:57.322784 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:57.323642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:57.323642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:57.324858 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:57.360443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:57.360746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.361465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:04:57.361465 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.361465 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:57.361465 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.361465 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.361465 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:57.361465 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.361465 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.362544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.363160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:57.363826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:57.364523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.365493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:57.365493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.365493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.365493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:57.365493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.365493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:57.533734 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:57.534413 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:57.534545 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:57.535774 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:57.536572 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:57.536825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.537727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:57.537727 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.537727 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:57.537727 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.537727 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.537727 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:57.537727 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.537727 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.539486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.540356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.540759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:57.541252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.541916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.541916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:57.541916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.541916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:57.543642 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:57.544360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:57.544511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:57.544662 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:04:57.545219 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:57.545508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.546164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:57.546164 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.546164 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:57.546164 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.546164 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.546164 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:57.546164 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.546164 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:57.547822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.548345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:57.548345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.548345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:57.548345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.548345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.548345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:57.548345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.548345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.549348 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:57.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.550067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.550067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:57.550067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.550067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.551472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:57.551957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.552591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.552591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:57.552591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.552591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:57.552591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:57.552591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:57.552591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:57.552591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:57.552945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:57.553172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.553764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:57.553764 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.553764 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:57.553764 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.553764 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.553764 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:57.553764 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.553764 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:04:57.555236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.555711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:04:57.555711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.555711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:04:57.555711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.555711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.555711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:04:57.555711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.555711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:57.555942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.556307 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:57.556716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:57.557257 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.557809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:57.557809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.557809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:57.557809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.557809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.557809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:57.557809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.557809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:04:57.561003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:04:57.561003 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:04:57.561234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:04:57.561234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:04:57.561483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:57.561666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:04:57.561770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.562125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:04:57.562125 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.562125 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:04:57.562125 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.562125 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.562125 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:04:57.562125 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.562125 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.563057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:04:57.563749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:04:57.564124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:04:57.564667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:04:57.565382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:04:57.565382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:04:57.565382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:02.323355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:02.324266 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:02.324418 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:02.325337 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:02.361714 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:02.362148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.363020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:02.363020 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.363020 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:02.363020 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.363020 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.363020 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:02.363020 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.363020 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.363869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.364504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:02.365084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:02.365816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.366653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:02.366653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.366653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.366653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:02.366653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.366653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:02.534522 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:02.537234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:02.537234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:02.538656 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:02.538904 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:02.539088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.539679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:02.539679 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.539679 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:02.539679 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.539679 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.539679 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:02.539679 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.539679 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.540452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.540701 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:02.540988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.540988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.541183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.541548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:02.541963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.541963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:02.544188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:02.544428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:02.544507 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:02.544600 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:02.545187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:02.545415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.545964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:02.545964 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.545964 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:02.545964 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.545964 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.545964 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:02.545964 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.545964 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:02.546652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.547100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:02.547100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.547100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:02.547100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.547100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.547100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:02.547100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.547100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.547315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:02.547581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.547864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.547864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:02.547864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.547864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.547975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:02.548337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.548733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.548733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:02.548733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.548733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:02.552044 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:02.552269 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:02.552519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:02.552519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:02.552808 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:02.553067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.553706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:02.553706 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.553706 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:02.553706 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.553706 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.553706 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:02.553706 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.553706 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:02.554262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.554580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:02.554580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.554580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:02.554580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.554580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.554580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:02.554580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.554580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:02.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.555320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.555605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:02.555605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.555605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.555835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:02.556261 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.556652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.556652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:02.556652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.556652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:02.561007 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:02.561428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:02.561554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:02.561809 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:02.562139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:02.562382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:02.562532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.563113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:02.563113 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.563113 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:02.563113 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.563113 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.563113 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:02.563113 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.563113 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.564061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:02.564379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.564632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:02.565180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:02.565544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:02.565934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:02.565934 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:07.324139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:07.325190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:07.325452 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:07.327061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:07.367849 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:07.368411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.369381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:07.369381 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.369381 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:07.369381 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.369381 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.369381 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:07.369381 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.369381 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.370283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.371000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:07.371695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:07.372528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.373135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:07.373135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.373135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.373135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:07.373135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.373135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:07.536690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:07.537042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:07.537186 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:07.538531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:07.538806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:07.538969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.539687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:07.539687 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.539687 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:07.539687 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.539687 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.539687 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:07.539687 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.539687 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.541603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.542135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:07.542636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.542636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.542728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.543416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:07.544125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.544125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:07.544686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:07.545498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:07.545659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:07.545830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:07.546398 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:07.546600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.547318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:07.547318 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.547318 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:07.547318 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.547318 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.547318 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:07.547318 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.547318 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:07.549115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.549853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:07.549853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.549853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:07.549853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.549853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.549853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:07.549853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.549853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.550680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:07.551191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.551764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.551764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:07.551764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.551764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.552579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:07.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.553987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.553987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:07.553987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.553987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:07.553987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:07.553987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:07.553987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:07.553987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:07.554336 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:07.554403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.554883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:07.554883 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.554883 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:07.554883 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.554883 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.554883 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:07.554883 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.554883 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:07.555951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.556521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:07.556521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.556521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:07.556521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.556521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.556521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:07.556521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.556521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:07.556901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.557308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.557687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:07.557687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.557687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.558574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:07.559175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.559697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.559697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:07.559697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.559697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:07.561472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:07.561852 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:07.561934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:07.562124 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:07.562685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:07.563090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:07.563403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.564168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:07.564168 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.564168 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:07.564168 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.564168 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.564168 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:07.564168 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.564168 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.565126 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:07.565548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.565869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:07.566497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:07.566981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:07.567466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:07.567466 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:12.324974 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:12.325922 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:12.325922 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:12.326973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:12.363036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:12.363630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.364614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:12.364614 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.364614 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:12.364614 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.364614 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.364614 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:12.364614 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.364614 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.365657 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.366364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:12.367133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:12.368149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.369015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:12.369015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.369015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.369015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:12.369015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.369015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:12.537427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:12.537937 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:12.538071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:12.539166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:12.539540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:12.539788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.540681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:12.540681 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.540681 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:12.540681 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.540681 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.540681 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:12.540681 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.540681 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.542009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.542644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.543179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:12.543953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.544851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.544851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:12.544851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.544851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:12.545405 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:12.545761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:12.545939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:12.546321 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:12.546773 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:12.546998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.547671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:12.547671 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.547671 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:12.547671 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.547671 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.547671 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:12.547671 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.547671 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:12.549173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:12.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:12.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:12.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.549704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.550643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:12.550994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.551365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.551365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:12.551365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.551365 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.552381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:12.552943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.553486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.553486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:12.553486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.553486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:12.553687 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:12.553995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:12.554063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:12.554140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:12.554409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:12.554592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:12.554941 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.554941 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:12.554941 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.554941 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.554941 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:12.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.554941 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:12.555561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.555972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:12.555972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.555972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:12.555972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.555972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.555972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:12.555972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.555972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:12.556287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.556651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.557119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:12.557119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.557119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.557207 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:12.557682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.558158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.558158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:12.558158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.558158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:12.561886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:12.562379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:12.562476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:12.562670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:12.562964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:12.563144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:12.563288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.564873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:12.564873 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.564873 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:12.564873 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.564873 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.564873 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:12.564873 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.564873 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.565917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:12.566297 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.566669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:12.566988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:12.567496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:12.567957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:12.567957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:17.325647 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:17.326463 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:17.326584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:17.327887 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:17.364055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:17.364535 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.365414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:17.365414 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.365414 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:17.365414 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.365414 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.365414 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:17.365414 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.365414 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.366185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.366877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:17.367550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:17.368371 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.369332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:17.369332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.369332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.369332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:17.369332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.369332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:17.538397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:17.539037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:17.540151 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:17.541340 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:17.541699 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:17.542022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.542995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:17.542995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.542995 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:17.542995 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.542995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.542995 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:17.542995 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.542995 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.544541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.545070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:17.545694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.545694 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.546395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.547188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:17.547664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.547664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:17.547664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:17.547664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:17.547664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:17.547664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:17.547957 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:17.548395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.549237 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:17.549237 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.549237 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:17.549237 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.549237 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.549237 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:17.549237 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.549237 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:17.550157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.550948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:17.550948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.550948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:17.550948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.550948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.550948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:17.550948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.550948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.551177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:17.551576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.551952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.551952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:17.551952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.551952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.552067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:17.552574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.553272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.553272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:17.553272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.553272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:17.554174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:17.554518 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:17.554629 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:17.554778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:17.555106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:17.555423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.555888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:17.555888 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.555888 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:17.555888 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.555888 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.555888 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:17.555888 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.555888 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:17.556897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.557326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:17.557326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.557326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:17.557326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.557326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.557326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:17.557326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.557326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:17.558117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.558580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.559020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:17.559020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.559020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.559437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:17.559988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.560499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.560499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:17.560499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.560499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:17.562502 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:17.562902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:17.563067 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:17.563322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:17.564205 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:17.564517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:17.564668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.565227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:17.565227 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.565227 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:17.565227 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.565227 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.565227 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:17.565227 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.565227 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.567006 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:17.567369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.567711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:17.568130 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:17.568613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:17.569076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:17.569076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:22.326523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:22.327632 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:22.327632 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:22.328511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:22.363632 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:22.364020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.365045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:22.365045 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.365045 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:22.365045 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.365045 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.365045 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:22.365045 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.365045 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.365864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.366558 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:22.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:22.368027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.369019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:22.369019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.369019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.369019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:22.369019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.369019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:22.539120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:22.539923 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:22.540081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:22.541379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:22.541840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:22.542122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.543057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:22.543057 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.543057 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:22.543057 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.543057 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.543057 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:22.543057 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.543057 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.544613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.545279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:22.545925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.545925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.546867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.547741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:22.548644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.548644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:22.548644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:22.548644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:22.548644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:22.548644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:22.548644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:22.549032 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.549534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:22.549534 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.549534 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:22.549534 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.549534 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.549534 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:22.549534 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.549534 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:22.550631 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.551420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:22.551926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.552327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:22.552327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:22.552327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.552327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:22.552327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.552327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:22.552327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.552327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.552327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:22.552327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.552327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.552646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:22.553160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.553798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.553798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:22.553798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.553798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:22.554610 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:22.554858 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:22.554939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:22.555061 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:22.555382 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:22.555597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.556015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:22.556015 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.556015 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:22.556015 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.556015 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.556015 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:22.556015 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.556015 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:22.556750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.557283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:22.557283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.557283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:22.557283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.557283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.557283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:22.557283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.557283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:22.557627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.558066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.558611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:22.558611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.558611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.559202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:22.559733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.560240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.560240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:22.560240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.560240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:22.563079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:22.563559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:22.563706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:22.563966 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:22.564442 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:22.564806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:22.565040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.565850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:22.565850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.565850 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:22.565850 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.565850 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.565850 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:22.565850 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.565850 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.567073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:22.567478 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.567814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:22.568172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:22.568964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:22.569846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:22.569846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:27.327257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:27.328037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:27.328130 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:27.329488 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:27.363762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:27.364159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.365174 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:27.365174 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.365174 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:27.365174 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.365174 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.365174 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:27.365174 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.365174 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.366591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.366969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:27.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.367309 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:27.367759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.368295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.369142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:27.369142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.369142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:27.539865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:27.540972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:27.540972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:27.541573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:27.541986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:27.542350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.543503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:27.543503 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.543503 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:27.543503 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.543503 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.543503 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:27.543503 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.543503 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.545104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.545771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:27.546821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.546821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.547089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.547911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:27.548942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.548942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:27.548942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:27.548942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:27.548942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:27.548942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:27.548942 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:27.548942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.549642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:27.549642 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.549642 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:27.549642 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.549642 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.549642 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:27.549642 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.549642 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:27.550647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.551528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:27.552280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.552884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:27.553578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.554295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:27.554925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:27.555231 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:27.555408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:27.555585 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:27.556043 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:27.556453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.557196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:27.557196 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.557196 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:27.557196 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.557196 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.557196 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:27.557196 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.557196 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:27.558548 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.558886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:27.558886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.558886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:27.558886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.558886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.558886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:27.558886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.558886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:27.560092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.560444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.560720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:27.560720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.560720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.562060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:27.562812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.563446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.563446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:27.563446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.563446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:27.563676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:27.564436 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:27.564579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:27.564850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:27.565224 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:27.565396 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:27.565504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.566003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:27.566003 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.566003 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:27.566003 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.566003 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.566003 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:27.566003 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.566003 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:27.568020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.568446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:27.568669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:27.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:27.570016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:27.570016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:32.328112 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:32.329191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:32.329191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:32.330145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:32.367127 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:32.367602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.368560 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:32.368560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.368560 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:32.368560 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.368560 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.368560 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:32.368560 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.368560 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.369710 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.370145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:32.370647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.370647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:32.370760 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.371316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.371928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:32.371928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.371928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:32.540634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:32.541161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:32.541323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:32.542742 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:32.543033 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:32.543161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.544021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:32.544021 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.544021 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:32.544021 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.544021 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.544021 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:32.544021 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.544021 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.545852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.546389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:32.546806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.546806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.547606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.548113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:32.548688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.548688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:32.548688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:32.548688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:32.548688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:32.548688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:32.549055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:32.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.550125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:32.550125 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.550125 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:32.550125 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.550125 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.550125 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:32.550125 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.550125 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:32.550876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.551402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:32.551402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.551402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:32.551402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.551402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.551402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:32.551402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.551402 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.551647 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:32.551973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:32.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.552337 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.552635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:32.553260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.553835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.553835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:32.553835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.553835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:32.555409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:32.555685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:32.555761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:32.555900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:32.556179 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:32.556431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.556852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:32.556852 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.556852 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:32.556852 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.556852 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.556852 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:32.556852 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.556852 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:32.557603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.557982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:32.557982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.557982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:32.557982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.557982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.557982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:32.557982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.557982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:32.558458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.558845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.559276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:32.559276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.559276 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.559515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:32.559972 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.560511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.560511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:32.560511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.560511 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:32.564260 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:32.564860 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:32.565018 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:32.565317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:32.565669 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:32.565889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:32.566031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.566589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:32.566589 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.566589 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:32.566589 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.566589 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.566589 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:32.566589 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.566589 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.568155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:32.568576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.568922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:32.569823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:32.570444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:32.571099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:32.571099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:37.328666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:37.329582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:37.329934 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:37.331433 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:37.369754 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:37.369754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.370960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:37.370960 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.370960 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:37.370960 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.370960 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.370960 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:37.370960 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.370960 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.372273 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.373244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:37.373908 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:37.374771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.377380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:37.377380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.377380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.377380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:37.377380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.377380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:37.542574 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:37.543689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:37.543689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:37.544411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:37.544846 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:37.545125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.546712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:37.546712 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.546712 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:37.546712 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.546712 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.546712 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:37.546712 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.546712 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.548264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.548673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:37.549317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.549317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.550467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.551048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:37.551636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.551636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:37.551636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:37.551636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:37.551636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:37.551636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:37.551636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:37.551636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.551932 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:37.551932 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.551932 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:37.551932 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.551932 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.551932 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:37.551932 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.551932 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:37.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.553403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:37.553403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.553403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:37.553403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.553403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.553403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:37.553403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.553403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.553827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:37.554326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.554994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:37.555456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:37.556691 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:37.557141 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:37.557141 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:37.557141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.557141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:37.557141 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.557141 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:37.557141 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.557141 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.557141 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:37.557141 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.557141 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:37.557797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.558294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:37.558294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.558294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:37.558294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.558294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.558294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:37.558294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.558294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:37.559024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.559640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:37.560929 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:37.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.562897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:37.562897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.562897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:37.562897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.562897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.562897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:37.562897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.562897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:37.565070 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:37.565738 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:37.565884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:37.566135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:37.566644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:37.566850 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:37.566969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.567463 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:37.567463 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.567463 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:37.567463 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.567463 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.567463 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:37.567463 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.567463 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.568909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:37.569354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.569706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:37.570521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:37.570992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:37.571475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:37.571475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:42.329655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:42.331548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:42.331548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:42.333062 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:42.335376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=434 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 14:05:42.368301 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:42.368788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.369966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:42.369966 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.369966 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:42.369966 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.369966 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.369966 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:42.369966 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.369966 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.371400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.371745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:42.372057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.372057 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:42.372708 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.373094 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.373533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:42.373533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.373533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:42.543268 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:42.543828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:42.543964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:42.545388 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:42.545695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:42.545871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.546577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:42.546577 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.546577 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:42.546577 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.546577 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.546577 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:42.546577 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.546577 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.548028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.548334 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:42.548634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.548634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.549538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.549919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:42.550279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.550279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:42.550985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:42.551369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:42.551531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:42.551693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:42.552060 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:42.552336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.552993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:42.552993 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.552993 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:42.552993 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.552993 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.552993 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:42.552993 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.552993 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:42.554024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.554427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:42.554427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.554427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:42.554427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.554427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.554427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:42.554427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.554427 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.555450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:42.555738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.556289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.556289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:42.556289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.556289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.557336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:42.557734 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.558135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.558135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:42.558135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.558135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:42.558135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:42.558135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:42.558135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:42.558135 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:42.558857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:42.559189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.559591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:42.559591 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.559591 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:42.559591 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.559591 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.559591 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:42.559591 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.559591 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:42.560423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.561016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:42.561016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.561016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:42.561016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.561016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.561016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:42.561016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.561016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:42.561461 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.561733 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.562004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:42.562004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.562004 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.562836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:42.563181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:42.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:42.565605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:42.565821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:42.565875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:42.566034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:42.566404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:42.566598 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:42.566711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.567175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:42.567175 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.567175 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:42.567175 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.567175 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.567175 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:42.567175 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.567175 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.568069 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:42.568399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.568650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:42.568851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:42.569335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:42.569709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:42.569709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:47.330435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:47.331413 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:47.331678 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:47.333319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:47.372654 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:47.374615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.374615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:47.374615 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.374615 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:47.374615 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.374615 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.374615 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:47.374615 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.374615 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.374968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.375501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:47.376167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:47.376799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.377884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:47.377884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.377884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.377884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:47.377884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.377884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:47.543946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:47.544675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:47.544854 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:47.546167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:47.546425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:47.546557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.547224 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:47.547224 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.547224 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:47.547224 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.547224 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.547224 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:47.547224 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.547224 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.548916 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.549428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:47.549935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.549935 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.550831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.551521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:47.552284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.552284 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:47.552284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:47.552284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:47.552284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:47.552284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:47.552546 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:47.552546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.552971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:47.552971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.552971 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:47.552971 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.552971 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.552971 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:47.552971 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.552971 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:47.553542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.554147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:47.554747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.555253 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:47.555817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.556534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:47.557993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:47.558702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:47.558702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:47.558702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:47.558854 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:47.559002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.559347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:47.559347 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.559347 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:47.559347 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.559347 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.559347 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:47.559347 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.559347 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:47.561410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.561685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:47.561941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:47.562179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.562179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:47.562179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.562179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:47.562179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.562179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.562179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:47.562179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.562179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.563111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:47.563726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.564363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.564363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:47.564363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.564363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:47.566088 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:47.566540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:47.566622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:47.566790 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:47.567142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:47.567384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:47.567530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.568285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:47.568285 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.568285 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:47.568285 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.568285 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.568285 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:47.568285 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.568285 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.569131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:47.569666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:47.570171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:47.570865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:47.571613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:47.571613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:47.571613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:52.331366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:52.332384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:52.332585 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:52.335180 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:52.369238 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:52.369592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.370586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:52.370586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.370586 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:52.370586 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.370586 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.370586 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:52.370586 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.370586 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.371667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.371970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:52.372260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.372260 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:52.373915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.374373 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.374763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:52.374763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.374763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:52.544884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:52.548356 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:52.548356 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:52.548356 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:52.548356 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:52.548356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.548944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:52.548944 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.548944 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:52.548944 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.548944 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.548944 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:52.548944 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.548944 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.550590 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.550967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:52.551268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.551268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.551808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.552165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:52.552528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.552528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:52.552528 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:52.552528 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:52.552528 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:52.552729 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:52.553106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:52.553398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.553980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:52.553980 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.553980 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:52.553980 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.553980 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.553980 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:52.553980 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.553980 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:52.554746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.555099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:52.555099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.555099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:52.555099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.555099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.555099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:52.555099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.555099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.555531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:52.556039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.556556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.556556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:52.556556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.556556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.556721 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:52.557142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.557520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.557520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:52.557520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.557520 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:52.558523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:52.559185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:52.559185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:52.559185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:52.559478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:52.559693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:52.562473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.562873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.563150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:52.563996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:52.564523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.564844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:52.565184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.565184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.565184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:52.565184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.565184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:52.566738 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:52.567077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:52.567134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:52.567316 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:52.568580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:52.568838 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:52.568999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.569846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:52.569846 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.569846 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:52.569846 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.569846 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.569846 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:52.569846 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.569846 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.570803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:52.571083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.571325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:52.571616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:52.572380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:52.572880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:52.572880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:57.332066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:57.333017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:57.333258 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:57.334787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:57.364889 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:57.365187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.365988 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:05:57.365988 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.365988 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:57.365988 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.365988 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.365988 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:57.365988 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.365988 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.367073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.367955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:57.368659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:57.369581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.371292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:57.371292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.371292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.371292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:57.371292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.371292 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:57.545496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:57.546109 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:57.546170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:57.547195 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:57.547466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:57.547600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.548242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:57.548242 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.548242 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:57.548242 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.548242 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.548242 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:57.548242 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.548242 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.549431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.549810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.550198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:57.550700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.551372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.551372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:57.551372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.551372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:57.552695 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:57.553129 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:57.553199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:57.553322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:05:57.553698 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:57.553886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.554378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:57.554378 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.554378 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:57.554378 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.554378 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.554378 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:57.554378 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.554378 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:57.555052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.556027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:57.556616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.557123 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:57.557605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.559078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:57.562143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:57.562143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:57.562143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:57.562143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:57.562143 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:57.562143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.562143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:57.562143 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.562346 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:57.562346 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.562346 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.562346 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:57.562346 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.562346 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:05:57.564481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.564795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:57.565093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:57.565352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:57.565606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.565957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:05:57.567287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:05:57.567472 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:05:57.567550 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:05:57.567653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:05:57.568036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:57.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:05:57.568475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.568930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:05:57.568930 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.568930 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:05:57.568930 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.568930 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.568930 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:05:57.568930 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.568930 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.569714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:05:57.570634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:05:57.571542 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:05:57.572195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:05:57.572603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:05:57.572603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:05:57.572603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:02.332632 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:02.334074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:02.334074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:02.334074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:02.367461 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:02.367738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.368393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:02.368393 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.368393 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:02.368393 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.368393 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.368393 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:02.368393 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.368393 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.369158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.369769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:02.370329 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:02.371755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.372743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:02.372743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.372743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.372743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:02.372743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.372743 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:02.546384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:02.547163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:02.547404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:02.548935 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:02.549388 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:02.549707 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.550824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:02.550824 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.550824 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:02.550824 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.550824 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.550824 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:02.550824 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.550824 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.552890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.553598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:02.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.554168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.554595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.555633 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:02.556607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.556607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:02.556607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:02.556607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:02.556607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:02.556607 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:02.557588 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:02.557926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.559900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:02.560648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:02.560812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:02.560996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:02.561575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:02.561575 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.561575 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:02.561575 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.561575 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.561575 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:02.561575 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.561575 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:02.562933 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.563638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:02.563638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.563638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:02.563638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.563638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.563638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:02.563638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.563638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:02.563638 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.564805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:02.565287 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.565666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.565666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:02.565666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.565666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.566026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:02.566544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.567112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.567112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:02.567112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.567112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:02.567112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.567322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:02.567322 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.567322 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:02.567322 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.567322 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.567322 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:02.567322 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.567322 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:02.569010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.569534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:02.569534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.569534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:02.569534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.569534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.569534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:02.569534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.569534 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:02.569534 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:02.571641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.572153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.572580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:02.572580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.572580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:02.572580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.573376 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:02.573841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.574318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.574318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:02.574318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.574318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:02.574318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:02.574318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:02.574487 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:02.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:02.574911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.575682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:02.575682 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.575682 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:02.575682 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.575682 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.575682 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:02.575682 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.575682 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.577149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:02.577554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.577875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:02.578144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:02.578605 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:02.579100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:02.579100 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:07.333287 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:07.334257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:07.335434 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:07.335525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:07.374072 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:07.374356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.374915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:07.374915 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.374915 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:07.374915 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.374915 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.374915 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:07.374915 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.374915 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.375687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.376487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:07.376980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:07.377544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.378132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:07.378132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.378132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.378132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:07.378132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.378132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:07.547153 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:07.547874 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:07.548075 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:07.549754 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:07.550243 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:07.550568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.551607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:07.551607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.551607 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:07.551607 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.551607 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.551607 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:07.551607 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.551607 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.552866 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.553510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:07.554867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.555766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.555766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:07.555766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.555766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:07.555766 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:07.556415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:07.556415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:07.556415 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:07.556933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:07.557199 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.557891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:07.557891 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.557891 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:07.557891 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.557891 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.557891 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:07.557891 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.557891 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:07.559313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.560165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:07.560165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.560165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:07.560165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.560165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.560165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:07.560165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.560165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.560692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:07.561092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.561518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.561518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:07.561518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.561518 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.562629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:07.563159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.563753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.563753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:07.563753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.563753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:07.563753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:07.563753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:07.563753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:07.563753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:07.564006 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:07.564240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.565001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:07.565001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.565001 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:07.565001 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.565001 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.565001 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:07.565001 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.565001 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:07.565752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.566258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:07.566258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.566258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:07.566258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.566258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.566258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:07.566258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.566258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:07.566664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.567070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.567473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:07.567473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.567473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.567689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:07.568234 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.568764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.568764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:07.568764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.568764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:07.569868 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:07.570262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:07.570349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:07.570501 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:07.571025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:07.571445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:07.571722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.572479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:07.572479 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.572479 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:07.572479 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.572479 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.572479 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:07.572479 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.572479 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.573704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:07.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.574537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:07.575044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:07.575589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:07.576110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:07.576110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:12.333902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:12.334886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:12.335128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:12.336729 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:12.378354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:12.378740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.379429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:12.379429 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.379429 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:12.379429 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.379429 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.379429 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:12.379429 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.379429 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.380555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.381158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:12.381792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:12.382528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.383497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:12.383497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.383497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.383497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:12.383497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.383497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:12.548005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:12.548770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:12.548770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:12.550804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:12.551139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:12.551353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.552131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:12.552131 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.552131 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:12.552131 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.552131 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.552131 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:12.552131 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.552131 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.553313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.553624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:12.553871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.553871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.554691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.555639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:12.556107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.556107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:12.556532 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:12.557328 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:12.557433 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:12.557580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:12.558198 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:12.558547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.559324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:12.559324 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.559324 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:12.559324 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.559324 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.559324 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:12.559324 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.559324 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:12.560938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.561528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:12.561528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.561528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:12.561528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.561528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.561528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:12.561528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.561528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.563440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:12.563945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.564242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.564242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:12.564242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.564242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.564591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:12.564973 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.565383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.565383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:12.565383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.565383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:12.565383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:12.565500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:12.566123 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:12.566391 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:12.567046 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:12.567552 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.568394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:12.568394 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.568394 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:12.568394 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.568394 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.568394 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:12.568394 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.568394 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:12.569409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.569737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:12.569737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.569737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:12.569737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.569737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.569737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:12.569737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.569737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:12.570341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.570618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.570894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:12.570894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.570894 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:12.570894 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.572015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:12.572393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.573248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.573248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:12.573248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.573248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:12.573248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:12.573248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:12.573248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:12.573248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:12.573630 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:12.573630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.574145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:12.574145 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.574145 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:12.574145 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.574145 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.574145 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:12.574145 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.574145 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.575074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:12.575464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.575848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:12.576059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:12.576544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:12.577052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:12.577052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:17.334542 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:17.335575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:17.335810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:17.337564 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:17.377765 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:17.377995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.378699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:17.378699 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.378699 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:17.378699 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.378699 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.378699 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:17.378699 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.378699 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.379784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.380440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:17.381073 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:17.381837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.382822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:17.382822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.382822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.382822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:17.382822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.382822 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:17.548768 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:17.549357 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:17.549496 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:17.550973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:17.551503 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:17.551869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.552922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:17.552922 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.552922 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:17.552922 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.552922 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.552922 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:17.552922 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.552922 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.554561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.555116 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:17.555728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.555728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.556452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.556913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:17.557500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.557500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:17.557500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:17.557500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:17.557500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:17.557500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:17.557865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:17.558192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.558926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:17.558926 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.558926 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:17.558926 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.558926 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.558926 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:17.558926 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.558926 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:17.560134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.560586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:17.560586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.560586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:17.560586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.560586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.560586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:17.560586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.560586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.561127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:17.561500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.561886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.561886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:17.561886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.561886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.563523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:17.563969 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.564596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.564596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:17.564596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.564596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:17.565539 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:17.566118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:17.566289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:17.566467 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:17.567086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:17.567559 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.568416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:17.568416 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.568416 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:17.568416 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.568416 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.568416 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:17.568416 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.568416 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:17.569807 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.570245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:17.570245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.570245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:17.570245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.570245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.570245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:17.570245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.570245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:17.572502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.572852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.573263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:17.573263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.573263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.576115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:17.576836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.577608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.577608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:17.577608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.577608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:17.577608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:17.577608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:17.577608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:17.577608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:17.578366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:17.578758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:17.579010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.580044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:17.580044 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.580044 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:17.580044 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.580044 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.580044 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:17.580044 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.580044 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.581479 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:17.581915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.582254 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:17.583125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:17.583679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:17.584267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:17.584267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:22.335248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:22.336116 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:22.336479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:22.338009 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:22.375723 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:22.375919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.376476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:22.376476 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.376476 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:22.376476 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.376476 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.376476 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:22.376476 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.376476 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.378096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.382306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:22.383953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:22.386459 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.387350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:22.387350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.387350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.387350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:22.387350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.387350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:22.549579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:22.550159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:22.550159 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:22.551379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:22.551812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:22.552315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.553056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:22.553056 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.553056 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:22.553056 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.553056 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.553056 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:22.553056 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.553056 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.554343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.555634 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.556280 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:22.561084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.562872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.562872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:22.562872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.562872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:22.562872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:22.562872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:22.562872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:22.562872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:22.562872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.563168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.563653 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.564071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:22.565746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.567061 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:22.568059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:22.569450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.570705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.571141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:22.571581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.571581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.571581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:22.571581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.571581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:22.571992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.572877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:22.573985 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:22.574516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.575015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:22.575015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.575015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:22.575015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.575015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.575015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:22.575015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.575015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:22.576814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:22.577345 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:22.577510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:22.577725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:22.578383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:22.578746 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:22.578981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.579802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:22.579802 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.579802 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:22.579802 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.579802 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.579802 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:22.579802 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.579802 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.581270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:22.581624 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.581953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:22.583153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:22.583618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:22.584051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:22.584051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:27.336045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:27.336751 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:27.336814 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:27.338551 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:27.374941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:27.375242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.375829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:27.375829 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.375829 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:27.375829 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.375829 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.375829 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:27.375829 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.375829 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.376962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.377317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:27.377615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.377615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:27.377919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.378394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.378918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:27.378918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.378918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:27.550573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:27.551126 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:27.551338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:27.552644 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:27.553040 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:27.553345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.554447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:27.554447 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.554447 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:27.554447 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.554447 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.554447 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:27.554447 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.554447 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.556837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.557467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:27.558023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.558023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.558978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.559986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:27.560623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.560623 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:27.560623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:27.560623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:27.560623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:27.560623 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:27.560831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:27.561090 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.561918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:27.561918 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.561918 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:27.561918 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.561918 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.561918 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:27.561918 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.561918 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:27.563251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.563796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:27.563796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.563796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:27.563796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.563796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.563796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:27.563796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.563796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.564669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:27.565099 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.565517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.565517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:27.565517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.565517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.565791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:27.566322 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.566875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.566875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:27.566875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.566875 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:27.566875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:27.566875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:27.566875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:27.567111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:27.567297 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:27.567525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.568054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:27.568054 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.568054 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:27.568054 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.568054 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.568054 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:27.568054 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.568054 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:27.568882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.569375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:27.569375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.569375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:27.569375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.569375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.569375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:27.569375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.569375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:27.570630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.571064 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.571464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:27.571464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.571464 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.572438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:27.572978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.573507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.573507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:27.573507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.573507 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:27.577229 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:27.578452 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:27.578595 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:27.578818 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:27.579369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:27.579726 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:27.579960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.580731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:27.580731 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.580731 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:27.580731 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.580731 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.580731 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:27.580731 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.580731 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.582493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:27.582931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.583263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:27.584191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:27.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:27.585437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:27.585437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:32.336821 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:32.337792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:32.337792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:32.339026 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:32.376351 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:32.376830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.377903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:32.377903 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.377903 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:32.377903 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.377903 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.377903 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:32.377903 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.377903 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.378833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.379556 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:32.380268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:32.380995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.381984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:32.381984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.381984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.381984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:32.381984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.381984 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:32.551165 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:32.551817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:32.551971 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:32.553428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:32.553834 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:32.554108 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.555830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:32.555830 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.555830 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:32.555830 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.555830 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.555830 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:32.555830 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.555830 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.557494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.558375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:32.559101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.559101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.559263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.560247 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:32.561513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.561513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:32.562620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:32.563227 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:32.563380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:32.563563 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:32.564189 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:32.564491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.566031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:32.566031 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.566031 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:32.566031 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.566031 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.566031 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:32.566031 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.566031 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:32.567339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.567944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:32.567944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.567944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:32.567944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.567944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.567944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:32.567944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.567944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.569304 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:32.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.569821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.569821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:32.569821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.569821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:32.569915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.572155 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:32.573041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.574039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.574039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:32.574039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.574039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:32.574039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:32.574039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:32.574039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:32.574039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:32.574039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.574570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:32.574570 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.574570 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:32.574570 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.574570 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.574570 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:32.574570 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.574570 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:32.574957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:32.576245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.577016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:32.577706 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:32.578499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:32.579489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:32.580185 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:32.580451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:32.580604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.581027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:32.581027 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.581027 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:32.581027 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.581027 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.581027 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:32.581027 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.581027 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.581845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:32.582150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.582428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:32.582642 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:32.583051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:32.583449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:32.583449 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:37.337343 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:37.338307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:37.338307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:37.339389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:37.372384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:37.372915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.374165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:37.374165 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.374165 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:37.374165 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.374165 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.374165 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:37.374165 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.374165 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.375148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.375514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:37.375913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.375913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:37.376316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.377820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.378374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:37.378374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.378374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:37.552078 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:37.554118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:37.554118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:37.554118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:37.554118 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:37.554370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.555187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:37.555187 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.555187 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:37.555187 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.555187 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.555187 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:37.555187 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.555187 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.556825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.557204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:37.557602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.557602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.558588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.558977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:37.559361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.559361 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:37.563096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:37.563536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:37.563664 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:37.563789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:37.565358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:37.565680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.566428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:37.566428 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.566428 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:37.566428 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.566428 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.566428 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:37.566428 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.566428 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:37.567286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.567608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:37.567608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.567608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:37.567608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.567608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.567608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:37.567608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.567608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.568435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:37.569036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.569423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:37.569859 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:37.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:37.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:37.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:37.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:37.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:37.570619 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:37.570820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:37.571247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:37.571247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:37.571658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:37.571820 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.573612 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:37.573612 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.573612 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:37.573612 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.573612 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.573612 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:37.573612 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.573612 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:37.574477 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.575335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:37.576005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:37.576355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.576355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:37.576355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.576355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:37.576355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.576355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.576355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:37.576355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.576355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.577058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:37.577499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.577998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.577998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:37.577998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.577998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:37.578728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:37.579305 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:37.579459 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:37.579694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:37.580047 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:37.580316 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:37.580458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.580994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:37.580994 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.580994 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:37.580994 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.580994 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.580994 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:37.580994 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.580994 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.582573 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:37.582904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.583252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:37.584293 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:37.584711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:37.585172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:37.585172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:42.338027 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:42.339262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:42.339262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:42.340483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:42.347842 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=241 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 14:06:42.388249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:42.388719 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.389712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:42.389712 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.389712 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:42.389712 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.389712 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.389712 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:42.389712 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.389712 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.390746 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.391154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:42.391561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.391561 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:42.391854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.392430 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.392987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:42.392987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.392987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:42.552710 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:42.553245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:42.553402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:42.554450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:42.554702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:42.554860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.555519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:42.555519 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.555519 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:42.555519 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.555519 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.555519 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:42.555519 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.555519 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.557065 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.557758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:42.558379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.558379 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.558968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.559818 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:42.560695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.560695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:42.563704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:42.564245 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:42.564390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:42.564749 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:42.565401 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:42.565627 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.566145 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:42.566145 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.566145 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:42.566145 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.566145 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.566145 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:42.566145 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.566145 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:42.567017 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.567538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:42.567538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.567538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:42.567538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.567538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.567538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:42.567538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.567538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.567831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:42.568295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.568671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.568671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:42.568671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.568671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.568857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:42.569328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.569852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.569852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:42.569852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.569852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:42.570871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:42.571147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:42.571286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:42.571403 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:42.571843 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:42.572093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.572635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:42.572635 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.572635 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:42.572635 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.572635 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.572635 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:42.572635 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.572635 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:42.573502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.573944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:42.573944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.573944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:42.573944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.573944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.573944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:42.573944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.573944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:42.574358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.574731 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.575119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:42.575119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.575119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.575378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:42.575873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.576442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.576442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:42.576442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.576442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:42.579133 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:42.579483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:42.579576 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:42.579733 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:42.580353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:42.580758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:42.580993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.581697 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:42.581697 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.581697 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:42.581697 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.581697 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.581697 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:42.581697 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.581697 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.582712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:42.583097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.583437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:42.583687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:42.584242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:42.584727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:42.584727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:47.338747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:47.339581 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:47.339581 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:47.340468 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:47.379635 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:47.380038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:47.380882 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.380882 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:47.380882 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.380882 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.380882 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:47.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.381798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.382416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:47.382993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:47.383725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.384689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:47.384689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.384689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.384689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:47.384689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.384689 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:47.553389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:47.553952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:47.554040 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:47.555081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:47.555469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:47.555717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.556713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:47.556713 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.556713 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:47.556713 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.556713 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.556713 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:47.556713 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.556713 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.557528 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.558110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.558813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:47.559622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.560783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.560783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:47.560783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.560783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:47.565079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:47.565079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:47.565079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:47.565079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:47.565531 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:47.565851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.566600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:47.566600 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.566600 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:47.566600 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.566600 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.566600 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:47.566600 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.566600 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:47.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.568864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:47.568864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.568864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:47.568864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.568864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.568864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:47.568864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.568864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.569097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:47.569474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.569844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.569844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:47.569844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.569844 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.570039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:47.570524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.571124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.571124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:47.571124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.571124 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:47.571259 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:47.571708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:47.571918 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:47.572055 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:47.572542 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:47.572887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.573576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:47.573576 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.573576 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:47.573576 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.573576 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.573576 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:47.573576 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.573576 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:47.574823 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.575455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:47.575455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.575455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:47.575455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.575455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.575455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:47.575455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.575455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:47.576467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.576977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.577523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:47.577523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.577523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.578472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:47.579249 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.579987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.579987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:47.579987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.579987 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:47.579987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:47.579987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:47.579987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:47.580412 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:47.580555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:47.580875 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:47.581089 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:47.583385 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.583385 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:47.583385 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.583385 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.583385 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:47.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.584011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:47.584546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:47.585052 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:47.585714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:47.586493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:47.586493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:47.586493 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:52.339511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:52.340582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:52.340582 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:52.341517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:52.380556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:52.380834 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.381385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:52.381385 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.381385 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:52.381385 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.381385 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.381385 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:52.381385 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.381385 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.382266 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.382956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:52.383372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:52.383821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.384382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:52.384382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.384382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.384382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:52.384382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.384382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:52.554128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:52.554758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:52.554910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:52.555950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:52.556222 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:52.556601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.557339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:52.557339 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.557339 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:52.557339 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.557339 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.557339 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:52.557339 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.557339 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.558942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.559692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.560469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:52.561415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.562407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.562407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:52.562407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.562407 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:52.564732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:52.565102 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:52.565189 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:52.565476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:52.565852 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:52.566146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.566860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:52.566860 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.566860 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:52.566860 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.566860 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.566860 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:52.566860 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.566860 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:52.567718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.568271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:52.568271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.568271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:52.568271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.568271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.568271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:52.568271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.568271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.568525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:52.568888 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.569332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.569332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:52.569332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.569332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.569622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:52.570104 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.570805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.570805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:52.570805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.570805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:52.571724 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:52.572603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:52.572603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:52.572603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:52.573138 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:52.573452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:52.577269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.577960 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.578438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:52.578785 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:52.579134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.579615 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:52.580120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.580120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.580120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:52.580120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.580120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:52.580272 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:52.580564 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:52.580686 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:52.580858 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:52.582048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:52.582048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:52.582048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.585506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:52.585506 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.585506 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:52.585506 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.585506 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.585506 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:52.585506 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.585506 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.586652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:52.587087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.587465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:52.587762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:52.588242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:52.588780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:52.588780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:57.340081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:57.341122 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:57.341122 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:57.342579 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:57.375081 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:57.375332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.375998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:06:57.375998 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.375998 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:57.375998 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.375998 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.375998 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:57.375998 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.375998 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.376782 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.377470 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:57.379078 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:57.380882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.381804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:57.381804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.381804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.381804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:57.381804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.381804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:57.554871 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:57.555362 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:57.555473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:57.556425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:57.556791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:57.557063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.557865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:57.557865 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.557865 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:57.557865 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.557865 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.557865 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:57.557865 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.557865 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.558640 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.559163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.559671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:57.560393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.561087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.561087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:57.561087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.561087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:57.565174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:57.565604 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:57.565736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:57.565890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:06:57.566628 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:57.566940 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.567698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:57.567698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.567698 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:57.567698 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.567698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.567698 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:57.567698 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.567698 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:57.569070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.569850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:57.569850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.569850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:57.569850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.569850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.569850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:57.569850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.569850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.570497 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:57.570868 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.571421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.571421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:57.571421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.571421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.572285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:57.572938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.573482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.573482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:57.573482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.573482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:57.573482 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:57.573482 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:57.573482 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:57.573690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:57.574162 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:57.574546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.575229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:57.575229 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.575229 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:57.575229 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.575229 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.575229 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:57.575229 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.575229 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:06:57.576637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.577091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:06:57.577091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.577091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:06:57.577091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.577091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.577091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:06:57.577091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.577091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:57.577533 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.577874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.578294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:57.578294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.578294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.578588 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:57.579063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.579699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.579699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:57.579699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.579699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:06:57.580627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:06:57.581080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:06:57.581257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:06:57.581498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:06:57.581943 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:57.582326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:06:57.582541 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.583204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:06:57.583204 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.583204 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:06:57.583204 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.583204 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.583204 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:06:57.583204 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.583204 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.584738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:06:57.585139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.585494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:06:57.586555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:06:57.587029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:06:57.587519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:06:57.587519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:02.341328 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:02.342257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:02.342257 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:02.343349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:02.371452 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:02.371801 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.372512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:02.372512 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.372512 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:02.372512 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.372512 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.372512 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:02.372512 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.372512 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.373250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.373680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:02.374067 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:02.374572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.375101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:02.375101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.375101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.375101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:02.375101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.375101 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:02.555793 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:02.556073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:02.556164 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:02.557147 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:02.557653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:02.557924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.560842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:02.560842 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.560842 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:02.560842 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.560842 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.560842 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:02.560842 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.560842 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.561783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.563382 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.563981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:02.567418 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.567905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.567905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:02.567905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.567905 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:02.568273 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:02.568411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:02.568411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:02.568627 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:02.568878 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:02.569083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.569604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:02.569604 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.569604 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:02.569604 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.569604 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.569604 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:02.569604 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.569604 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:02.570355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.571039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:02.571039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.571039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:02.571039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.571039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.571039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:02.571039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.571039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.571313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:02.571699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.572062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.572062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:02.572062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.572062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.572413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:02.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.575275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.575275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:02.575275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.575275 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:02.575275 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:02.575275 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:02.575275 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:02.575275 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:02.575826 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:02.575826 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.576093 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:02.576093 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.576093 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:02.576093 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.576093 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.576093 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:02.576093 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.576093 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:02.576821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.577248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:02.577248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.577248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:02.577248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.577248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.577248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:02.577248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.577248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:02.581712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.582035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.582347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:02.582347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.582347 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.583944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:02.584482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.584970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.584970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:02.584970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.584970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:02.585933 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:02.586188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:02.586285 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:02.586436 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:02.586958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:02.587387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:02.587669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.588290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:02.588290 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.588290 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:02.588290 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.588290 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.588290 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:02.588290 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.588290 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.588845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:02.589248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:02.589582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:02.590323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:02.590871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:02.590871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:02.590871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:07.341924 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:07.343042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:07.343042 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:07.344086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:07.383244 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:07.383602 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.384506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:07.384506 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.384506 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:07.384506 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.384506 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.384506 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:07.384506 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.384506 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.385316 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.385650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:07.385930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.385930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:07.386377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.386763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.387160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:07.387160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.387160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:07.556402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:07.556867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:07.557002 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:07.558017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:07.558344 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:07.558589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.559502 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:07.559502 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.559502 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:07.559502 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.559502 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.559502 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:07.559502 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.559502 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.560978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.561440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:07.561876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.561876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.562503 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.563025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:07.563610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.563610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:07.568460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:07.568732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:07.568807 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:07.568896 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:07.569227 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:07.569443 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.569879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:07.569879 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.569879 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:07.569879 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.569879 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.569879 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:07.569879 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.569879 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:07.571019 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.571620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:07.571620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.571620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:07.571620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.571620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.571620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:07.571620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.571620 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.572308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:07.572765 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.573229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.573229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:07.573229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.573229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.573974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:07.574521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.575132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.575132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:07.575132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.575132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:07.575132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:07.575132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:07.575132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:07.575132 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:07.575455 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:07.575837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.576514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:07.576514 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.576514 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:07.576514 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.576514 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.576514 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:07.576514 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.576514 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:07.577763 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.578303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:07.578303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.578303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:07.578303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.578303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.578303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:07.578303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.578303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:07.579327 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.579713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.580117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:07.580117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.580117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.581143 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:07.581666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.582167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.582167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:07.582167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.582167 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:07.586376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:07.586620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:07.586679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:07.586810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:07.587060 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:07.587291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:07.587421 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.587825 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:07.587825 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.587825 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:07.587825 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.587825 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.587825 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:07.587825 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.587825 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.588638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:07.589159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:07.589774 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:07.590574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:07.591524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:07.591524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:07.591524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:12.342573 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:12.343817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:12.343817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:12.344900 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:12.381857 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:12.382336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.383191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:12.383191 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.383191 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:12.383191 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.383191 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.383191 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:12.383191 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.383191 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.384024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.384641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:12.385291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:12.385991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.386878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:12.386878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.386878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.386878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:12.386878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.386878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:12.557017 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:12.557469 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:12.557658 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:12.558795 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:12.559411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:12.559611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.560578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:12.560578 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.560578 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:12.560578 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.560578 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.560578 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:12.560578 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.560578 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.562118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.564547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.565591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:12.566271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.567079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.567079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:12.567079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.567079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:12.568915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:12.569219 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:12.569318 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:12.569402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:12.569820 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:12.570063 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.570584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:12.570584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.570584 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:12.570584 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.570584 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.570584 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:12.570584 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.570584 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:12.571294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.572532 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:12.573808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.574131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:12.574499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.575028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:12.575570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:12.575570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:12.575570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:12.575570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:12.575973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:12.576447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.577162 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:12.577162 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.577162 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:12.577162 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.577162 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.577162 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:12.577162 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.577162 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:12.578336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.578791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:12.578791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.578791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:12.578791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.578791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.578791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:12.578791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.578791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:12.579750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.580156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.580577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:12.580577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.580577 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.581025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:12.581685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.582233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.582233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:12.582233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.582233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:12.586799 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:12.587049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:12.587167 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:12.587306 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:12.587569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:12.587840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:12.587979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.588599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:12.588599 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.588599 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:12.588599 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.588599 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.588599 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:12.588599 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.588599 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.589855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:12.590352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.590775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:12.591345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:12.591879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:12.592405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:12.592405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:17.343166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:17.344001 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:17.344001 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:17.344643 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:17.378860 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:17.379240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.379941 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:17.379941 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.379941 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:17.379941 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.379941 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.379941 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:17.379941 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.379941 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.380899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.381582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:17.382235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:17.382949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.385343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:17.385343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.385343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.385343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:17.385343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.385343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:17.557655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:17.558145 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:17.558349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:17.559693 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:17.560071 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:17.560353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.561154 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:17.561154 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.561154 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:17.561154 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.561154 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.561154 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:17.561154 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.561154 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.562850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.563674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:17.564003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.564003 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.565106 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.565661 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:17.566114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.566114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:17.569419 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:17.569704 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:17.569808 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:17.569883 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:17.570234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:17.570496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.570965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:17.570965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.570965 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:17.570965 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.570965 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.570965 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:17.570965 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.570965 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:17.571726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.572189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:17.572189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.572189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:17.572189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.572189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.572189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:17.572189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.572189 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.572900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:17.573727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.574191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:17.574607 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.575077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:17.575077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:17.575077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.575077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:17.575077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.575077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:17.575077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.575077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.575077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:17.575077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.575077 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:17.575366 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:17.575688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:17.575839 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:17.576008 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:17.576542 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:17.576893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.577632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:17.577632 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.577632 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:17.577632 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.577632 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.577632 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:17.577632 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.577632 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:17.578983 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.579481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:17.579481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.579481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:17.579481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.579481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.579481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:17.579481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.579481 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:17.580680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.581023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.581544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:17.581544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.581544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.582585 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:17.586356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.587353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.587353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:17.587353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.587353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:17.587353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:17.587724 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:17.587724 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:17.587724 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:17.588248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:17.588745 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:17.589045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.589893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:17.589893 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.589893 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:17.589893 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.589893 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.589893 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:17.589893 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.589893 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.590792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:17.591476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:17.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:17.593472 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:17.594716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:17.594716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:17.594716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:22.343987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:22.344769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:22.344769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:22.345639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:22.378041 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:22.378652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.379650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:22.379650 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.379650 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:22.379650 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.379650 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.379650 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:22.379650 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.379650 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.380540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.381138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:22.381714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:22.382381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.383164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:22.383164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.383164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.383164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:22.383164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.383164 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:22.558411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:22.559016 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:22.559158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:22.560163 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:22.560542 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:22.560861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.561698 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:22.561698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.561698 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:22.561698 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.561698 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.561698 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:22.561698 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.561698 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.562578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.563156 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.563846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:22.565391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.566537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.566537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:22.566537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.566537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:22.569877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:22.570641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:22.570641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:22.570641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:22.571090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:22.571431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.572395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:22.572395 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.572395 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:22.572395 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.572395 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.572395 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:22.572395 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.572395 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:22.573157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.573522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:22.573522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.573522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:22.573522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.573522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.573522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:22.573522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.573522 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.574012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:22.574295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.574572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.574572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:22.574572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.574572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.575037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:22.575406 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.575791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.575791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:22.575791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.575791 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:22.575791 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:22.576196 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:22.576374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:22.576556 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:22.577190 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:22.577571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.578448 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:22.578448 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.578448 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:22.578448 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.578448 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.578448 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:22.578448 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.578448 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:22.579784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.580272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:22.580272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.580272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:22.580272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.580272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.580272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:22.580272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.580272 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:22.581431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.581816 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.582176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:22.582176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.582176 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.582482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:22.582967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.583570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.583570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:22.583570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.583570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:22.587671 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:22.588076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:22.588076 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:22.588199 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:22.588616 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:22.589080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:22.589456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.590227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:22.590227 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.590227 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:22.590227 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.590227 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.590227 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:22.590227 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.590227 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.591572 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:22.592009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.592440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:22.593217 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:22.593699 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:22.594350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:22.594350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:27.344753 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:27.345500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:27.345500 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:27.346752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:27.384577 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:27.385011 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.385867 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:27.385867 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.385867 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:27.385867 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.385867 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.385867 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:27.385867 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.385867 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.387192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.387790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:27.388320 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:27.388918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.389636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:27.389636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.389636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.389636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:27.389636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.389636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:27.559638 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:27.560206 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:27.560206 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:27.561375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:27.561762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:27.562010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.562996 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:27.562996 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.562996 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:27.562996 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.562996 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.562996 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:27.562996 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.562996 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.564036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.565002 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.565440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:27.566012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.567317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.567317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:27.567317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.567317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:27.570335 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:27.570689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:27.570787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:27.570893 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:27.571253 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:27.571482 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.572557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:27.572557 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.572557 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:27.572557 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.572557 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.572557 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:27.572557 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.572557 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:27.573144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.574134 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:27.574980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.575770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:27.576469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.577097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:27.577681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:27.577681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:27.577681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:27.577681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:27.577681 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:27.577681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.578043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:27.578043 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.578043 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:27.578043 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.578043 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.578043 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:27.578043 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.578043 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:27.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.579961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:27.579961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.579961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:27.579961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.579961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.579961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:27.579961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.579961 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:27.580971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.581803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:27.581803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.581803 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.582691 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:27.583177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.583771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.583771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:27.583771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.583771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:27.588156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:27.588489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:27.588564 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:27.588865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:27.589390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:27.589811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:27.590098 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.591246 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:27.591246 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.591246 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:27.591246 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.591246 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.591246 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:27.591246 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.591246 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.591850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:27.592296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.592742 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:27.593388 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:27.594256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:27.595163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:27.595163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:32.367592 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:32.367989 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:32.368188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:32.369714 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:32.411944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:32.412226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.412799 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:32.412799 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.412799 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:32.412799 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.412799 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.412799 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:32.412799 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.412799 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.413539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.414138 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:32.414913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:32.415592 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.416495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:32.416495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.416495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.416495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:32.416495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.416495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:32.560188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:32.560758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:32.560861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:32.562034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:32.562479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:32.562739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.563641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:32.563641 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.563641 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:32.563641 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.563641 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.563641 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:32.563641 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.563641 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.564863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.565468 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.566165 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:32.566915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.567951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.567951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:32.567951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.567951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:32.570812 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:32.571353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:32.571443 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:32.571555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:32.572093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:32.572445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.573102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:32.573102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.573102 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:32.573102 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.573102 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.573102 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:32.573102 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.573102 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:32.573964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.574539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:32.574539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.574539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:32.574539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.574539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.574539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:32.574539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.574539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.575010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:32.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.575802 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:32.576228 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:32.576769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:32.577785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:32.577785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:32.577785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:32.578191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:32.578579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.579256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:32.579256 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.579256 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:32.579256 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.579256 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.579256 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:32.579256 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.579256 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:32.581405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.582736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:32.583331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:32.583880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:32.584849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.585662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:32.588622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:32.588980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:32.589177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:32.589374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:32.589902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:32.590346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:32.590610 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.591444 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:32.591444 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.591444 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:32.591444 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.591444 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.591444 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:32.591444 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.591444 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.592446 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:32.592755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.593037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:32.593413 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:32.593787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:32.594252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:32.594252 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:37.368338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:37.369404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:37.369404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:37.369787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:37.408792 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:37.409236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.409851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:37.409851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.409851 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:37.409851 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.409851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.409851 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:37.409851 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.409851 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.410917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.411296 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:37.412445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:37.413231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.413974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:37.413974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.413974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.413974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:37.413974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.413974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:37.561170 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:37.561538 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:37.561690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:37.562853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:37.563205 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:37.563516 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.564565 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:37.564565 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.564565 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:37.564565 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.564565 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.564565 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:37.564565 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.564565 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.566075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.566514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:37.566871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.566871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.567664 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.568310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:37.568805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.568805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:37.571391 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:37.571833 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:37.571972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:37.572128 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:37.572892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:37.573201 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.574135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:37.574135 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.574135 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:37.574135 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.574135 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.574135 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:37.574135 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.574135 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:37.575401 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.576465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:37.576465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.576465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:37.576465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.576465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.576465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:37.576465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.576465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.576878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:37.577688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.578808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:37.579714 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:37.580670 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.581387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.582037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:37.583142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.584060 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:37.585087 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.585377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.585736 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:37.589085 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:37.589282 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:37.589617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:37.589617 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:37.589853 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:37.590073 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:37.590220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:37.591305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.591945 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:37.592356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:37.592832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:37.593267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:37.593267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:37.593267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:37.593267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:37.593267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:37.593267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:37.593267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:37.593267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:37.593267 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:42.368985 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:42.370727 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:42.370727 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:42.371015 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:42.373388 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=239 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 14:07:42.406584 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:42.407053 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.408027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:42.408027 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.408027 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:42.408027 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.408027 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.408027 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:42.408027 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.408027 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.409046 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.409692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:42.410328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:42.411044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.411944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:42.411944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.411944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.411944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:42.411944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.411944 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:42.562720 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:42.563160 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:42.563354 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:42.564387 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:42.564744 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:42.564963 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.565474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:42.565474 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.565474 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:42.565474 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.565474 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.565474 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:42.565474 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.565474 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.569389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.570753 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:42.573027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.573967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.573967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:42.573967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.573967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:42.573967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:42.573967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:42.573967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:42.573967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:42.573967 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:42.574473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.574473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:42.574473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.574473 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:42.574473 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.574473 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.574473 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:42.574473 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.574473 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:42.574854 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.575695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:42.576390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.577366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:42.578508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.579949 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:42.580636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:42.580636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:42.580636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:42.580636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:42.580636 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:42.580636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.580831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:42.580831 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.580831 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:42.580831 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.580831 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.580831 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:42.580831 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.580831 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:42.581970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.582476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:42.582476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.582476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:42.582476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.582476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.582476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:42.582476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.582476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:42.583364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.583776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.584122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:42.584122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.584122 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.585033 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:42.585563 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.586013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.586013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:42.586013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.586013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:42.589513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:42.589830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:42.589986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:42.590220 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:42.590741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:42.591069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:42.591381 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.592295 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:42.592295 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.592295 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:42.592295 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.592295 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.592295 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:42.592295 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.592295 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.593759 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:42.594264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.594613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:42.595431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:42.597091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:42.597975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:42.597975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:47.369757 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:47.370685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:47.370685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:47.371876 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:47.406931 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:47.407181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.407829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:47.407829 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.407829 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:47.407829 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.407829 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.407829 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:47.407829 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.407829 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.408656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.409341 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:47.410086 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:47.410632 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.411248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:47.411248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.411248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.411248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:47.411248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.411248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:47.563506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:47.564037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:47.564176 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:47.565136 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:47.565450 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:47.565679 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.566608 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:47.566608 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.566608 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:47.566608 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.566608 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.566608 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:47.566608 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.566608 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.567968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.568415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:47.568750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.568750 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.569601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.570157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:47.571314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.571314 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:47.572408 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:47.572831 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:47.572911 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:47.572994 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:47.573543 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:47.573978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.574656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:47.574656 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.574656 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:47.574656 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.574656 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.574656 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:47.574656 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.574656 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:47.575504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.576036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:47.576036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.576036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:47.576036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.576036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.576036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:47.576036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.576036 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.576303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:47.576667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.577039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.577039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:47.577039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.577039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.577279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:47.577815 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.578393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.578393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:47.578393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.578393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:47.579389 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:47.579863 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:47.579964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:47.580079 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:47.580673 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:47.581133 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.581896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:47.581896 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.581896 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:47.581896 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.581896 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.581896 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:47.581896 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.581896 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:47.583062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.583539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:47.583539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.583539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:47.583539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.583539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.583539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:47.583539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.583539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:47.584440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.584814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.585177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:47.585177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.585177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.585683 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:47.586288 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.586915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.586915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:47.586915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.586915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:47.590045 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:47.590307 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:47.590369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:47.590544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:47.590988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:47.591369 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:47.591583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.592222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:47.592222 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.592222 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:47.592222 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.592222 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.592222 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:47.592222 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.592222 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.593221 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:47.593641 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.594018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:47.594274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:47.594769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:47.595289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:47.595289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:52.370560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:52.371405 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:52.371405 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:52.372690 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:52.407569 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:52.408000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.408910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:52.408910 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.408910 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:52.408910 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.408910 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.408910 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:52.408910 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.408910 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.409695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.410172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:52.410715 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:52.411344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.412114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:52.412114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.412114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.412114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:52.412114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.412114 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:52.564184 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:52.564649 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:52.564787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:52.565708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:52.565961 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:52.566129 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.566876 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:52.566876 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.566876 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:52.566876 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.566876 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.566876 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:52.566876 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.566876 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.573891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.575258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:52.575986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.575986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:52.575986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.576153 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.576917 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:52.577770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.577770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:52.577770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:52.577770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:52.577770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:52.577770 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:52.577770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.577770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:52.577770 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.577770 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:52.577770 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:52.578792 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:52.580305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:52.581737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.582438 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.583120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.584487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:52.585158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.585158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:52.585158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.585158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.585158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:52.585158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.585158 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:52.585158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:52.585158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:52.585158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:52.585158 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:52.585483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:52.587294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.588091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.588567 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:52.588904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:52.589251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.589682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:52.590141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.590141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.590141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:52.590141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.590141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:52.590358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:52.590958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:52.590958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:52.590958 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:52.591296 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:52.591646 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:52.591858 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.593346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:52.593346 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.593346 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:52.593346 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.593346 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.593346 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:52.593346 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.593346 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.593652 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:52.593897 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.594149 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:52.594575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:52.594914 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:52.595290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:52.595290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:57.371249 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:57.373036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:57.373036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:57.373390 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:57.408091 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:57.408580 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.409091 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:07:57.409091 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.409091 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:57.409091 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.409091 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.409091 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:57.409091 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.409091 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.410117 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.411125 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:57.411586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:57.412132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.412824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:57.412824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.412824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.412824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:57.412824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.412824 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:57.564796 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:57.565380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:57.565527 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:57.566694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:57.567049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:57.567414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.568391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:57.568391 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.568391 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:57.568391 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.568391 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.568391 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:57.568391 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.568391 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.570042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.570650 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:57.571195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.571195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.571773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.572583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:57.573428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.573428 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:57.575436 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:57.575830 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:57.575954 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:57.576086 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:07:57.576653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:57.576970 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.577709 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:57.577709 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.577709 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:57.577709 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.577709 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.577709 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:57.577709 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.577709 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:57.578547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.579059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:57.579059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.579059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:57.579059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.579059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.579059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:57.579059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.579059 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.579339 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:57.579704 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.580092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.580092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:57.580092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.580092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.580300 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:57.580827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.581442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.581442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:57.581442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.581442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:57.581679 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:57.581892 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:57.582021 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:57.582169 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:57.582675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:57.583058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.583889 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:57.583889 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.583889 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:57.583889 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.583889 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.583889 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:57.583889 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.583889 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:07:57.585132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.585595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:07:57.585595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.585595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:07:57.585595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.585595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.585595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:07:57.585595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.585595 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:57.586582 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.586982 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.587395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:57.587395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.587395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.588416 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:57.588896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.589429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.589429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:57.589429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.589429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:07:57.590741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:07:57.591275 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:07:57.591432 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:07:57.591674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:07:57.592035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:57.592333 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:07:57.592508 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.593015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:07:57.593015 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.593015 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:07:57.593015 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.593015 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.593015 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:07:57.593015 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.593015 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.594636 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:07:57.595029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.595370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:07:57.596172 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:07:57.596716 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:07:57.597268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:07:57.597268 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:02.371851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:02.372776 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:02.372776 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:02.373543 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:02.412708 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:02.413043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.414241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:02.414241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.414241 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:02.414241 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.414241 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.414241 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:02.414241 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.414241 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.415510 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.416140 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:02.416852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:02.417638 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.418770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:02.418770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.418770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.418770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:02.418770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.418770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:02.565513 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:02.566004 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:02.566168 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:02.567284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:02.567697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:02.567998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.568829 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:02.568829 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.568829 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:02.568829 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.568829 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.568829 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:02.568829 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.568829 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.569878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.570147 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:02.570429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.570429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.570958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.571332 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:02.571717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.571717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:02.603806 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:02.604788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:02.604788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:02.604788 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:02.605523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:02.605523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:02.606103 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:02.606409 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:02.606647 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:02.607338 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:02.608127 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:02.608458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:02.609191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.609364 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:02.609728 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:02.610240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 14:08:02.610240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.610240 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:02.610240 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.610240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.610240 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:02.610240 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.610240 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:02.611720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:02.612173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.612173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:02.612173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.612173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:02.612173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.612173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.612173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:02.612173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.612173 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.613066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:02.613433 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.613798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.613798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:02.613798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.613798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:02.614797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.615335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:02.615796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.615796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:02.615796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.615796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.615796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:02.615796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.615796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:02.616617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.616851 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:08:02.616851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.616851 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:02.616851 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.616851 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.616851 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:02.616851 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.616851 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:02.618085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:02.618539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.618539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:02.618539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.618539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:02.618539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.618539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.618539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:02.618539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.618539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:02.619029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_ha_failover failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.619029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:02.619029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:02.619029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:02.619029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.619029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:02.619029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:02.619029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:02.619029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:02.619029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.619029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:02.619424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.619770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.619770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:02.619770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.619770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.620171 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:02.620739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.621226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.621226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:02.621226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.621226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:02.621226 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:02.621226 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:02.621226 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.622238 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:02.622238 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.622238 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:02.622238 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.622238 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.622238 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:02.622238 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.622238 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.014 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.623893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:02.624317 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.624680 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:02.625639 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:02.626730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:02.627855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:02.627855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:07.372458 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:07.373309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:07.373309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:07.374478 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:07.408896 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:07.409196 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.410175 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:07.410175 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.410175 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:07.410175 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.410175 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.410175 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:07.410175 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.410175 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.411112 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.411609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:07.411957 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:07.412452 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.412943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:07.412943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.412943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.412943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:07.412943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.412943 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:07.566276 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:07.566689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:07.566828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:07.568036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:07.568413 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:07.568666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.569546 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:07.569546 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.569546 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:07.569546 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.569546 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.569546 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:07.569546 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.569546 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.571072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.571780 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:07.572344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.572344 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.572863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.573667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:07.574498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.574498 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:07.604758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:07.605117 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:07.605303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:07.605439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:07.606070 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:07.606659 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:07.606946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.607825 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:07.608542 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:07.608752 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:07.609380 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:07.609836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.610332 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:07.610766 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:07.610988 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:07.611291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:07.612519 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 14:08:07.612519 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.612519 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:07.612519 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.612519 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.612519 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:07.612519 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.612519 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:07.612931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:08:07.612931 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.612931 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:07.612931 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.612931 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.612931 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:07.612931 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.612931 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:07.613320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:07.614771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:07.615179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.615179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:07.615179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.615179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:07.615179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.615179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.615179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:07.615179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.615179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:07.616474 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.616827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.616827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:07.616827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.616827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:07.616827 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:07.618026 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.618465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Traceback (most recent call last):
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.618913 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] OVN maintenance task update_mac_aging_settings failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.619649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.619998 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:07.620392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.620392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-8e52a737-7a8f-4c1d-af8c-15aee874ddf4 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:07.620618 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.621040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.621574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:07.621574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.621574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:07.640367 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:07.641107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.641107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:07.641107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.641107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:07.641107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.641107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.641107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:07.641107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.641107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:07.642512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_ha_failover failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.642512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:07.642512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:07.642512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:07.642512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.642512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:07.642512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:07.642512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:07.642512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:07.642512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.642512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:07.642853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.643283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.643283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:07.643283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.643283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:07.644554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:07.645045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:07.645616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:07.645616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:07.645616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:07.645616 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:12.373144 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:12.373993 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:12.374090 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:12.375291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:12.408683 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:12.409010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.409974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:12.409974 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.409974 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:12.409974 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.409974 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.409974 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:12.409974 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.409974 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.410971 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.411659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:12.412354 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:12.413185 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.414540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:12.414540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.414540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.414540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:12.414540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.414540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:12.566947 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:12.567426 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:12.567648 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:12.568674 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:12.568894 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:12.569044 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.569777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:12.569777 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.569777 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:12.569777 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.569777 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.569777 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:12.569777 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.569777 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.571467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.572068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:12.572656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.572656 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.573074 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.573886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:12.574392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.574392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:12.605173 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:12.605800 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:12.605991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:12.606187 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:12.606991 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:12.607363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.608191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 14:08:12.608191 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.608191 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:12.608191 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.608191 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.608191 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:12.608191 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.608191 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:12.609764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:12.610557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.610557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:12.610557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.610557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:12.610557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.610557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.610557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:12.610557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.610557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.611187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:12.611805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.612429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.612429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:12.612429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.612429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:12.612965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.613674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:12.614475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.614475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:12.614475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.614475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.614475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:12.614475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.614475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:12.642481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:12.642481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:12.642665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:12.642852 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:12.643025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:12.643464 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:12.643701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:12.643995 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:12.644740 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:12.645146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.646306 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:08:12.646306 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.646306 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:12.646306 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.646306 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.646306 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:12.646306 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.646306 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:12.647773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:12.648202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.648202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:12.648202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.648202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:12.648202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.648202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.648202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:12.648202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.648202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:12.648202 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:12.649938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.649938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:12.649938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:12.649938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:12.649938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.649938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:12.649938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:12.649938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:12.649938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:12.649938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.649938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:12.650793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.651732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.651732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:12.651732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.651732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:12.651732 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.652311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:12.653184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Traceback (most recent call last):
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.653955 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] OVN maintenance task update_mac_aging_settings failed after 0.010 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.654900 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.655486 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:12.656010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.656010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-27461d51-2f70-4a4d-8c49-d78af75360e7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:12.656456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:12.656955 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:12.657485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:12.657485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:12.657485 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:17.373865 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:17.374965 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:17.375057 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:17.377895 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:17.418039 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:17.418310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.419043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:17.419043 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.419043 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:17.419043 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.419043 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.419043 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:17.419043 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.419043 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.419951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.420360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:17.420741 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:17.421305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.421910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:17.421910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.421910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.421910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:17.421910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.421910 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:17.567639 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:17.568191 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:17.568319 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:17.569499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:17.569926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:17.570222 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.571540 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:17.571540 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.571540 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:17.571540 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.571540 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.571540 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:17.571540 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.571540 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.573187 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.573881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:17.574553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.574553 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.574997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.575712 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:17.576294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.576294 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:17.605765 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:17.606560 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:17.606701 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:17.606866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:17.607428 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:17.608021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.608904 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 14:08:17.608904 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.608904 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:17.608904 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.608904 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.608904 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:17.608904 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.608904 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:17.609965 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:17.610836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.610836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:17.610836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.610836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:17.610836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.610836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.610836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:17.610836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.610836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.611353 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:17.612177 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:17.612852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.613959 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.614880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:17.642946 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:17.643489 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:17.644286 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:17.644874 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:17.645353 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:17.645548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:17.645864 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:17.646174 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:17.647018 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:17.647568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.648641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:17.649183 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:17.649183 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.649183 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:17.649183 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.649183 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.649183 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:17.649183 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.649183 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:17.649548 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:17.650645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.651442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:17.651442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.651442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:17.651442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.651442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.651442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:17.651442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.651442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:17.651442 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:17.652043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.652658 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:17.653338 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:17.654037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.654902 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.655637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.656356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:17.657236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:17.657925 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:17.658576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:17.658576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:17.658576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:17.658576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:17.658576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:17.658576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:22.374647 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:22.375727 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:22.375727 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:22.376926 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:22.409586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:22.410012 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.410885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:22.410885 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.410885 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:22.410885 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.410885 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.410885 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:22.410885 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.410885 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.411964 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.412604 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:22.413315 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:22.414045 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.415054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:22.415054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.415054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.415054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:22.415054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.415054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:22.568289 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:22.568605 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:22.568688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:22.569689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:22.570063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:22.570346 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.571085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:22.571085 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.571085 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:22.571085 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.571085 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.571085 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:22.571085 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.571085 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.572127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.572751 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.573336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:22.574047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.575080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.575080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:22.575080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.575080 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:22.606384 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:22.606952 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:22.607161 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:22.607404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:22.607980 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:22.608299 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.609070 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 14:08:22.609070 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.609070 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:22.609070 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.609070 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.609070 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:22.609070 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.609070 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:22.610456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:22.611020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.611020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:22.611020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.611020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:22.611020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.611020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.611020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:22.611020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.611020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.611828 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:22.612291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.612727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.612727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:22.612727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.612727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:22.613531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.614039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:22.614495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.614495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:22.614495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.614495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.614495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:22.614495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.614495 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:22.643810 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:22.644025 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:22.644302 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:22.644525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:22.644912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:22.645321 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:22.645609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.646339 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:22.646936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:08:22.646936 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.646936 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:22.646936 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.646936 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.646936 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:22.646936 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.646936 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:22.647407 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:22.648767 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:22.649456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.649456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:22.649456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.649456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:22.649456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.649456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.649456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:22.649456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.649456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:22.649456 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:22.650398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.650398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:22.650398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:22.650398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:22.650398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.650398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:22.650398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:22.650398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:22.650398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:22.650398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.650398 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:22.650956 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:22.651938 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:22.652587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:22.653429 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.654335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:22.654335 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.654335 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:22.654335 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.654335 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.654335 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:22.654335 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.654335 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.655356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:22.656235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.656600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:22.656979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:22.657484 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:22.657966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:22.657966 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:27.375721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:27.376524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:27.376524 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:27.377146 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:27.411487 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:27.411788 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.412501 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:27.412501 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.412501 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:27.412501 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.412501 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.412501 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:27.412501 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.412501 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.413285 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.413727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:27.414148 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:27.414662 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.415467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:27.415467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.415467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.415467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:27.415467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.415467 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:27.569134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:27.569416 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:27.569568 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:27.570815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:27.571232 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:27.571550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.572414 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:27.572414 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.572414 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:27.572414 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.572414 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.572414 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:27.572414 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.572414 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.574696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:27.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.575387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.575869 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.576515 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:27.577000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.577000 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:27.607035 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:27.607666 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:27.607899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:27.608124 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:27.610586 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:27.610907 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.611832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 14:08:27.611832 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.611832 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:27.611832 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.611832 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.611832 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:27.611832 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.611832 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:27.613324 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:27.614215 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.615010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.615693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:27.617622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.618439 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:27.619360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.619360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:27.619360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.619360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.619360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:27.619360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.619360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:27.644514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:27.645509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:27.645694 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:27.645866 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:27.646376 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:27.646984 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:27.647622 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:27.647776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.648969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:27.648969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:27.649153 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:27.649370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:27.649895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.650192 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:08:27.650192 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.650192 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:27.650192 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.650192 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.650192 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:27.650192 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.650192 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:27.651325 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:27.652688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:27.653500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.654141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:27.654141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:27.654141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.654141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:27.654141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.654141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:27.654141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.654141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.654141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:27.654141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.654141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.655024 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:27.655397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Traceback (most recent call last):
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.655770 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] OVN maintenance task update_mac_aging_settings failed after 0.008 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.657360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.657643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:27.657880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.657880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-0fedae9b-89b6-48a4-b99a-b250f0dfe04a None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:27.659264 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:27.659568 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:27.659927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:27.659927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:27.659927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:32.376613 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:32.377602 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:32.377692 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:32.379517 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:32.419910 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:32.420384 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.421223 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:32.421223 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.421223 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:32.421223 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.421223 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.421223 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:32.421223 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.421223 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.422335 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.422954 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:32.423740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.423740 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:32.424705 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.426475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.427755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:32.427755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.427755 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:32.569915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:32.570442 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:32.570554 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:32.571626 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:32.571861 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:32.572025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.572674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:32.572674 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.572674 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:32.572674 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.572674 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.572674 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:32.572674 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.572674 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.574159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.574895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:32.575536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.575536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.575671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.576494 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:32.577441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.577441 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:32.607997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:32.608557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:32.608746 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:32.608936 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:32.609545 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:32.609856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.611083 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Traceback (most recent call last):
Apr 21 14:08:32.611083 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.611083 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:32.611083 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.611083 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.611083 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:32.611083 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.611083 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:32.612651 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:32.614035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.614035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:32.614035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.614035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:32.614035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.614035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.614035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:32.614035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.614035 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.614766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:32.615579 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-f9820ddc-e761-4af5-8b28-2813d8c37284 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:32.616088 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.616853 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.617378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:32.645536 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:32.646005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:32.646111 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:32.646327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:32.646891 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:32.647435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.648479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:32.648936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:08:32.648936 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.648936 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:32.648936 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.648936 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.648936 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:32.648936 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.648936 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:32.650611 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:32.651506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.651506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:32.651506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.651506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:32.651506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.651506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.651506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:32.651506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.651506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:32.651506 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:32.652622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.652622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:32.652622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:32.652622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:32.652622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.652622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:32.652622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:32.652622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:32.652622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:32.652622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.652622 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:32.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.654005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.654005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:32.654005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.654005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:32.654005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.654899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:32.655635 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.656139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.656139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:32.656139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.656139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:32.656139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:32.656139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:32.656139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:32.656139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.656589 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:32.656589 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.656589 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:32.656589 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.656589 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.656589 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:32.656589 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.656589 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.657795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:32.658181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.658735 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:32.658884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:32.659378 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:32.659922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:32.659922 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:37.377370 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:37.379299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:37.379299 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:37.380343 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:37.426641 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:37.426993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.428016 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:37.428016 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.428016 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:37.428016 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.428016 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.428016 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:37.428016 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.428016 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.429269 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.429681 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:37.430066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.430066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:37.430462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.430981 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.431527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:37.431527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.431527 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:37.570877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:37.571139 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:37.571425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:37.572653 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:37.573262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:37.573475 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.574523 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:37.574523 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.574523 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:37.574523 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.574523 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.574523 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:37.574523 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.574523 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.576270 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.576860 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:37.577536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.577536 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.577890 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.578369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:37.578874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.578874 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:37.608655 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:37.609074 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:37.609192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:37.609326 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:37.609949 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:37.610391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.611163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:37.611163 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.611163 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:37.611163 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.611163 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.611163 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:37.611163 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.611163 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:37.612028 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.612794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:37.612794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.612794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:37.612794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.612794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.612794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:37.612794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.612794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.613135 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:37.613752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.614892 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:37.615649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.618230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:37.618230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:37.618230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.618230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:37.618230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.618230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:37.618230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.618230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.618230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:37.618230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.618230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:37.645977 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:37.646763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:37.646987 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:37.647284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:37.647992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:37.648471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.649471 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:08:37.649471 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.649471 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:37.649471 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.649471 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.649471 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:37.649471 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.649471 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:37.651351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:37.651798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.651798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:37.651798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.651798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:37.651798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.651798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.651798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:37.651798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.651798 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:37.651798 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:37.653271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.653271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:37.653271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:37.653271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:37.653271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.653271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:37.653271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:37.653271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:37.653271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:37.653271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.653271 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:37.653674 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.654096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.654096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:37.654096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.654096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:37.654096 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.655027 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:37.655583 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.656258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.656258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:37.656258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.656258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:37.656258 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:37.656258 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:37.656473 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:37.656775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:37.657037 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.657923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:37.657923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.657923 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:37.657923 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.657923 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.657923 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:37.657923 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.657923 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.659768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:37.660181 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.660682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:37.661544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:37.662039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:37.662806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:37.662806 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:42.379028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:42.380273 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:42.380273 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:42.381107 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:42.391925 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=240 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 14:08:42.423335 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:42.423613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.424359 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:42.424359 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.424359 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:42.424359 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.424359 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.424359 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:42.424359 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.424359 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.426539 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.427343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:42.427978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.427978 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:42.428364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.429241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.430235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:42.430235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.430235 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:42.571785 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:42.572140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:42.572316 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:42.572992 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:42.573183 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:42.573343 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.573899 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:42.573899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.573899 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:42.573899 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.573899 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.573899 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:42.573899 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.573899 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.574918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.575613 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.576274 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:42.577054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.578318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.578318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:42.578318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.578318 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:42.609395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:42.609756 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:42.609877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:42.610084 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:42.610894 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:42.611198 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.612020 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:42.612020 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.612020 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:42.612020 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.612020 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.612020 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:42.612020 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.612020 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:42.613370 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.614256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:42.614256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.614256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:42.614256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.614256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.614256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:42.614256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.614256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.614952 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:42.615775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.617058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:42.617769 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.618601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:42.618601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:42.618601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.618601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:42.618601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.618601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:42.618601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.618601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.618601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:42.618601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.618601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:42.646665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:42.647823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:42.647823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:42.647983 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:42.648912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:42.649374 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.650321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:08:42.650321 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.650321 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:42.650321 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.650321 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.650321 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:42.650321 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.650321 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:42.651913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:42.652352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.652352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:42.652352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.652352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:42.652352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.652352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.652352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:42.652352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.652352 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:42.653732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.653732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:42.653732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:42.653732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:42.653732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.653732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:42.653732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:42.653732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:42.653732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:42.653732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.653732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:42.654043 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.654425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.654425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:42.654425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.654425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:42.654425 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.655591 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:42.656146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.656764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.656764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:42.656764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.656764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:42.656764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:42.656764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:42.656764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:42.656764 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:42.657029 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:42.657227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.657864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:08:42.657864 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.657864 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:42.657864 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.657864 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.657864 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:42.657864 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.657864 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.658980 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.659391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:42.659738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.659738 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:42.660113 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:42.660584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:42.661079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:42.661079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:42.661079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:47.379852 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:47.380721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:47.380721 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:47.381150 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:47.418540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:47.418946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.419927 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:47.419927 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.419927 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:47.419927 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.419927 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.419927 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:47.419927 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.419927 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.421531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.422342 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:47.423058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.423058 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:47.423878 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.424732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.425659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:47.425659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.425659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:47.572580 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:47.572998 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:47.573140 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:47.574246 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:47.574564 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:47.574856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.575754 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:47.575754 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.575754 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:47.575754 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.575754 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.575754 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:47.575754 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.575754 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.577233 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.577930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:47.578570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.578570 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.579056 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.580278 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:47.581262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.581262 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:47.610028 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:47.610430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:47.610543 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:47.610782 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:47.611479 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:47.611909 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.612665 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:47.612665 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.612665 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:47.612665 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.612665 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.612665 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:47.612665 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.612665 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:47.614139 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.615504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:47.615504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.615504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:47.615504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.615504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.615504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:47.615504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.615504 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.615958 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:47.616513 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.617079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.617079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:47.617079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.617079 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.617286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:47.618010 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.618848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.618848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:47.618848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.618848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:47.647374 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:47.647886 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:47.648022 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:47.648099 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:47.648689 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:47.649191 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.651437 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:08:47.651437 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.651437 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:47.651437 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.651437 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.651437 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:47.651437 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.651437 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:47.652150 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:47.653166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:47.654084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:47.654928 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:47.655748 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.656835 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:47.657705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:47.657705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:47.657705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:47.657705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:47.657921 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:47.658336 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:47.658645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.661547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:47.661547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.661547 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:47.661547 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.661547 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.661547 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:47.661547 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.661547 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.663405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:47.663912 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.664399 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:47.665939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:47.666833 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:47.667649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:47.667649 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:52.380481 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:52.383559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:52.383559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:52.383559 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:52.418855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:52.419380 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.420186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:52.420186 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.420186 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:52.420186 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.420186 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.420186 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:52.420186 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.420186 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.421144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.421762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:52.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:52.423132 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.424009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:52.424009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.424009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.424009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:52.424009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.424009 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:52.573288 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:52.573823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:52.573823 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:52.574855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:52.575234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:52.575524 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.576377 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:52.576377 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.576377 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:52.576377 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.576377 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.576377 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:52.576377 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.576377 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.577458 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.578179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.578903 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:52.579775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.580887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.580887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:52.580887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.580887 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:52.610525 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:52.610902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:52.611037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:52.611197 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:52.611736 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:52.612022 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.612924 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:52.612924 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.612924 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:52.612924 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.612924 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.612924 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:52.612924 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.612924 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:52.613974 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.614831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:52.614831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.614831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:52.614831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.614831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.614831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:52.614831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.614831 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.615283 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:52.615855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.616726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.616726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:52.616726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.616726 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.616923 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:52.617855 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.619051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.619051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:52.619051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.619051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:52.647929 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:52.648320 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:52.648504 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:52.648813 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:52.649324 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:52.649693 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.650394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:08:52.650394 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.650394 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:52.650394 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.650394 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.650394 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:52.650394 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.650394 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:52.651236 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:52.652023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.652023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:52.652023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.652023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:52.652023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.652023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.652023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:52.652023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.652023 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:52.652529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.652529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:52.652529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:52.652529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:52.652529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.652529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:52.652529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:52.652529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:52.652529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:52.652529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.652529 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:52.653107 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:52.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:52.654727 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.655687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:52.655687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:52.655687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.655687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:52.655687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.655687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:52.655687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.655687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.655687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:52.655687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.655687 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:52.656776 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:52.657225 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:52.657457 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:52.657758 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:52.658498 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:52.658880 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:52.659096 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.659830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 14:08:52.659830 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.659830 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:52.659830 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.659830 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.659830 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:52.659830 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.659830 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.661179 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.661837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:52.662465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.662465 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:52.662783 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:52.663819 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:52.664845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:52.664845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:52.664845 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:57.381080 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:57.381755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:57.381755 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:57.383778 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:57.429959 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:57.430411 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.431319 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:08:57.431319 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.431319 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:57.431319 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.431319 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.431319 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:57.431319 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.431319 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.432085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.432393 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:57.432671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.432671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:57.432841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.433242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.433666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:57.433666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.433666 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:57.574005 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:57.575066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:57.575066 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:57.575776 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:57.576262 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:57.576810 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:57.579770 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.580286 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:57.580648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:57.581072 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:57.581773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.581773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:57.581773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.581773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:57.581773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.581773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.581773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:57.581773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.581773 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:57.611193 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:57.611615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:57.611746 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:57.611902 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:08:57.612608 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:57.612913 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.613870 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:08:57.613870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.613870 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:57.613870 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.613870 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.613870 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:57.613870 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.613870 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:57.615232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.615617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:57.615617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.615617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:57.615617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.615617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.615617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:57.615617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.615617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.616725 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:57.616992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.617258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.617258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:57.617258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.617258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.619331 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:57.619842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.620397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.620397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:57.620397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.620397 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:57.648634 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:57.649077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:57.649189 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:57.649381 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:57.650133 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:57.650586 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.651986 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:08:57.651986 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.651986 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:57.651986 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.651986 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.651986 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:57.651986 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.651986 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:08:57.653291 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:08:57.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:08:57.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:08:57.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:08:57.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.654001 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:08:57.654936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.654936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:57.654936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:57.654936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:57.654936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.654936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:57.654936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:57.654936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:57.654936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:57.654936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.654936 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:57.655372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.655718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.655718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:57.655718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.655718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.656115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:57.656671 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.657603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.657603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:57.657603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.657603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:08:57.657603 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:08:57.657763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:08:57.657763 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:08:57.657972 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:08:57.658483 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:57.658893 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:08:57.659180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.659931 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 14:08:57.659931 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.659931 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:08:57.659931 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.659931 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.659931 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:08:57.659931 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.659931 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.660946 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.661305 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:08:57.661839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.661839 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:08:57.661994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:08:57.662451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:08:57.663369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:08:57.663369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:08:57.663369 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:02.381789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:02.382437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:02.382437 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:02.383063 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:02.416591 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:02.417068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.418358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:09:02.418358 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.418358 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:02.418358 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.418358 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.418358 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:02.418358 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.418358 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.419062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.419915 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:02.420669 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:02.421447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:02.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:02.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.422445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:02.574919 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:02.575557 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:02.575672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:02.577953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:02.577953 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:02.577953 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.578190 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:02.578190 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.578190 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:02.578190 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.578190 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.578190 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:02.578190 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.578190 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.579357 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.579730 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:02.580109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.580109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.580425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.580919 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:02.581424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.581424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:02.611716 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:02.612348 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:02.612558 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:02.612789 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:09:02.614375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:02.614375 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.615062 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:02.615062 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.615062 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:02.615062 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.615062 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.615062 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:02.615062 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.615062 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:02.616758 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.617629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:02.617629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.617629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:02.617629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.617629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.617629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:02.617629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.617629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.618768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:02.619737 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.620424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.620424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:02.620424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.620424 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.620776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:02.621629 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.622745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.622745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:02.622745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.622745 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:02.649093 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:02.649460 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:02.649621 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:02.649747 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:02.650725 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:02.651188 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.652170 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:09:02.652170 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.652170 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:02.652170 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.652170 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.652170 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:02.652170 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.652170 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:02.653092 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:02.653804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.653804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:02.653804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.653804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:02.653804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.653804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.653804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:02.653804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.653804 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:02.654202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.654202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:02.654202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:02.654202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:02.654202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.654202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:02.654202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:02.654202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:02.654202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:02.654202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.654202 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:02.654749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:02.655389 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:02.655993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.656797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:02.656797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:02.656797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.656797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:02.656797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.656797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:02.656797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.656797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.656797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:02.656797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.656797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:02.657769 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:02.658067 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:02.658172 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:02.658360 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:02.658936 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:02.659367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:02.659614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.660487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 14:09:02.660487 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.660487 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:02.660487 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.660487 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.660487 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:02.660487 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.660487 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.661390 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.661723 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:02.662051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.662051 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:02.662372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:02.662775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:02.663311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:02.663311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:02.663311 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:07.383252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:07.383252 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:07.383867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:07.384621 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:07.416997 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:07.417345 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.417995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:09:07.417995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.417995 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:07.417995 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.417995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.417995 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:07.417995 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.417995 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.418757 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.419194 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:07.419673 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:07.420159 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.420879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:07.420879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.420879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.420879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:07.420879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.420879 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:07.575706 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:07.576427 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:07.576555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:07.577996 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:07.578367 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:07.578703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.579873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:07.579873 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.579873 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:07.579873 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.579873 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.579873 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:07.579873 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.579873 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.581646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.582258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:07.582857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.582857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.583423 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.584184 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:07.585039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.585039 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:07.612587 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:07.613155 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:07.613350 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:07.613445 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:09:07.614358 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:07.614813 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.615896 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:07.615896 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.615896 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:07.615896 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.615896 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.615896 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:07.615896 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.615896 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:07.617251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.618195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:07.618195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.618195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:07.618195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.618195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.618195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:07.618195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.618195 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.618728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:07.619302 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.619886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.619886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:07.619886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.619886 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.620239 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:07.621013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.621962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.621962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:07.621962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.621962 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:07.649804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:07.651077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:07.651077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:07.651077 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:07.651702 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:07.652111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.653351 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:09:07.653351 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.653351 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:07.653351 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.653351 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.653351 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:07.653351 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.653351 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:07.654366 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:07.655015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.655015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:07.655015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.655015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:07.655015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.655015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.655015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:07.655015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.655015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:07.655921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.655921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:07.655921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:07.655921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:07.655921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.655921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:07.655921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:07.655921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:07.655921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:07.655921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.655921 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:07.656992 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:07.657997 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:07.661473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:07.662645 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:07.663048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:07.663048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:07.663048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:07.663048 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:07.663048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.663048 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 14:09:07.663048 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.663048 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:07.663048 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:07.663372 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.663794 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.664193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:07.664193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.664193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:07.665220 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:07.665695 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:07.666231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:07.666231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:07.666231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:12.383261 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:12.384036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:12.384036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:12.385036 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:12.417395 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:12.417766 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.418505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:09:12.418505 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.418505 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:12.418505 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.418505 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.418505 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:12.418505 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.418505 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.419805 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.420473 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:12.421075 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:12.421893 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.422827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:12.422827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.422827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.422827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:12.422827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.422827 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:12.576322 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:12.577049 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:12.577192 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:12.578665 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:12.579227 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:12.579601 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.580787 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:12.580787 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.580787 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:12.580787 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.580787 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.580787 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:12.580787 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.580787 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.584110 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.585141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:12.586025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.586025 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.586525 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.587391 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:12.588328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.588328 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:12.613177 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:12.613840 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:12.614019 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:12.614179 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:09:12.614780 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:12.615084 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.615857 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:12.615857 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.615857 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:12.615857 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.615857 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.615857 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:12.615857 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.615857 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:12.617245 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.618031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:12.618031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.618031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:12.618031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.618031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.618031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:12.618031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.618031 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.618574 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:12.619197 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.619873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.619873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:12.619873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.619873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.620244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:12.621068 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.621692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.621692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:12.621692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.621692 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:12.650420 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:12.651189 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:12.651439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:12.651663 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:12.652950 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:12.653505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.654882 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:09:12.654882 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.654882 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:12.654882 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.654882 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.654882 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:12.654882 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.654882 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:12.656728 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:12.657555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.657555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:12.657555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.657555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:12.657555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.657555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.657555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:12.657555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.657555 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:12.658229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.658229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:12.658229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:12.658229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:12.658229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.658229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:12.658229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:12.658229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:12.658229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:12.658229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.658229 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:12.658871 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.659594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.659594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:12.659594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.659594 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.660163 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:12.660732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.661303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.661303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:12.661303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.661303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:12.661303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:12.661303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:12.661303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:12.661303 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:12.661563 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:12.661817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:12.661979 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.662626 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:09:12.662626 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.662626 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:12.662626 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.662626 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.662626 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:12.662626 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.662626 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.663537 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.663968 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:12.664308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.664308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:12.664538 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:12.665227 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:12.665713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:12.665713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:12.665713 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:17.384383 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:17.385394 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:17.385493 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:17.386711 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:17.422411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:17.422861 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.423848 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:09:17.423848 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.423848 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:17.423848 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.423848 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.423848 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:17.423848 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.423848 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.424628 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.425308 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:17.427244 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:17.427977 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.428884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:17.428884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.428884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.428884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:17.428884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.428884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:17.577009 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:17.577510 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:17.577790 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:17.579034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:17.579423 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:17.579682 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.580676 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:17.580676 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.580676 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:17.580676 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.580676 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.580676 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:17.580676 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.580676 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.582711 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.583409 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:17.584021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.584021 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.585400 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.586451 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:17.586994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.586994 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:17.613787 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:17.614248 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:17.614449 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:17.614609 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:09:17.615274 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:17.615557 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.616688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:17.616688 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.616688 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:17.616688 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.616688 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.616688 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:17.616688 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.616688 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:17.618071 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.619279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:17.619279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.619279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:17.619279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.619279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.619279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:17.619279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.619279 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.619762 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:17.620489 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.621200 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:17.622015 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.623038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:17.623038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:17.623038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.623038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:17.623038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.623038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:17.623038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.623038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.623038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:17.623038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.623038 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:17.651024 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:17.651495 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:17.651687 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:17.651944 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:17.652330 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:17.652551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.653109 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:09:17.653109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.653109 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:17.653109 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.653109 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.653109 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:17.653109 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.653109 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:17.654356 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:17.655142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.655142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:17.655142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.655142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:17.655142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.655142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.655142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:17.655142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.655142 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:17.656119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.656119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:17.656119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:17.656119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:17.656119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.656119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:17.656119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:17.656119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:17.656119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:17.656119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.656119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:17.656842 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.657562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.657562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:17.657562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.657562 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.658005 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:17.658832 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.659739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.659739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:17.659739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.659739 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:17.660912 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:17.661466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:17.661672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:17.661945 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:17.662404 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:17.662868 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:17.663146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.663995 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 14:09:17.663995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.663995 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:17.663995 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.663995 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.663995 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:17.663995 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.663995 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.665747 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.666157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:17.666587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.666587 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:17.667617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:17.668136 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:17.668863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:17.668863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:17.668863 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:22.385492 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:22.386466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:22.386466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:22.387804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:22.421355 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:22.421355 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.422250 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:09:22.422250 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.422250 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:22.422250 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.422250 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.422250 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:22.422250 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.422250 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.423603 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.424242 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:22.424948 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:22.426581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.427550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:22.427550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.427550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.427550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:22.427550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.427550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:22.577817 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:22.578166 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:22.578309 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:22.579514 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:22.579828 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:22.580085 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.580951 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:22.580951 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.580951 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:22.580951 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.580951 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.580951 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:22.580951 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.580951 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.582047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.582864 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:22.583455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.583455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.583551 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.584258 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:22.585013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.585013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:22.614236 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:22.614581 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:22.614775 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:22.614939 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:09:22.615615 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:22.615939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.617609 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:22.617609 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.617609 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:22.617609 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.617609 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.617609 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:22.617609 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.617609 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:22.618796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.619487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:22.619487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.619487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:22.619487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.619487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.619487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:22.619487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.619487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.620097 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:22.620644 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.621251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.621251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:22.621251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.621251 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.622118 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:22.622637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.623240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.623240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:22.623240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.623240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:22.651453 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:22.652411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:22.652411 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:22.652705 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:22.653466 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:22.653817 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.654752 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:09:22.654752 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.654752 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:22.654752 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.654752 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.654752 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:22.654752 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.654752 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:22.656076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:22.656790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.656790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:22.656790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.656790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:22.656790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.656790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.656790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:22.656790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.656790 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:22.657596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.657596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:22.657596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:22.657596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:22.657596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.657596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:22.657596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:22.657596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:22.657596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:22.657596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.657596 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:22.658383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.659076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.659076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:22.659076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.659076 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.659517 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:22.660415 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.661323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.661323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:22.661323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.661323 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:22.661323 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:22.661974 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:22.662183 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:22.662519 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:22.662986 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:22.663555 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:22.663891 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.664939 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 14:09:22.664939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.664939 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:22.664939 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.664939 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.664939 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:22.664939 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.664939 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.666581 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.666942 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:22.667310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.667310 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:22.668496 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:22.668950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:22.669483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:22.669483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:22.669483 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:27.386156 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:27.386855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:27.386855 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:27.387761 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:27.421565 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:27.421837 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.422576 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:09:27.422576 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.422576 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:27.422576 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.422576 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.422576 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:27.422576 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.422576 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.423531 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.423937 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:27.424364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.424364 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:27.424614 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.425029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.425500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:27.425500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.425500 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:27.578462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:27.579037 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:27.579152 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:27.580102 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:27.580379 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:27.580545 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.581161 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:27.581161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.581161 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:27.581161 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.581161 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.581161 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:27.581161 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.581161 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.582703 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.583385 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:27.583918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.583918 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.584593 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.585491 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:27.586447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.586447 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:27.614762 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:27.615095 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:27.615240 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:27.615375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:09:27.615973 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:27.616321 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.617180 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:27.617180 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.617180 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:27.617180 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.617180 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.617180 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:27.617180 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.617180 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:27.618440 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.619312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:27.619312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.619312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:27.619312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.619312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.619312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:27.619312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.619312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.619797 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:27.620505 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.621263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.621263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:27.621263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.621263 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.621455 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:27.622469 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.623360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.623360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:27.623360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.623360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:27.652182 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:27.652523 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:27.652624 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:27.652715 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:27.653183 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:27.653499 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.654119 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:09:27.654119 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.654119 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:27.654119 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.654119 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.654119 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:27.654119 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.654119 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:27.654852 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:27.655776 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:27.656487 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:27.657120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:27.657718 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.658648 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:27.661851 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:27.662397 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:27.662508 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:27.662729 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:27.663133 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:27.663435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:27.663575 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.664606 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 14:09:27.664606 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.664606 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:27.664606 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.664606 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.664606 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:27.664606 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.664606 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.665599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.665999 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:27.666350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.666350 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:27.666630 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:27.667186 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:27.667777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:27.667777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:27.667777 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:32.386872 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:32.387660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:32.387660 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:32.388898 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:32.425235 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:32.425521 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.426646 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:09:32.426646 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.426646 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:32.426646 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.426646 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.426646 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:32.426646 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.426646 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.427445 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.427796 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:32.428157 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:32.428578 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.429102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:32.429102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.429102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.429102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:32.429102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.429102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:32.579499 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:32.580106 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:32.580515 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:32.582434 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:32.582434 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:32.582434 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.583454 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:32.583454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.583454 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:32.583454 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.583454 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.583454 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:32.583454 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.583454 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.584993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.585696 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:32.586405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.586405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.586880 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.587926 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:32.588846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.588846 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:32.615438 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:32.615977 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:32.616113 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:32.616317 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:09:32.616879 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:32.617248 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.618042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:32.618042 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.618042 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:32.618042 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.618042 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.618042 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:32.618042 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.618042 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:32.619453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.619911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:32.619911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.619911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:32.619911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.619911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.619911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:32.619911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.619911 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.620930 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:32.621303 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.621659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.621659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:32.621659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.621659 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.622967 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:32.623599 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.624144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.624144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:32.624144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.624144 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:32.652738 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:32.653378 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:32.653620 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:32.653899 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:32.654540 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:32.654901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.655877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:09:32.655877 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.655877 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:32.655877 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.655877 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.655877 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:32.655877 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.655877 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:32.657453 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:32.658127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.658127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:32.658127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.658127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:32.658127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.658127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.658127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:32.658127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.658127 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:32.659255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.659255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:32.659255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:32.659255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:32.659255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.659255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:32.659255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:32.659255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:32.659255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:32.659255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.659255 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:32.659991 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:32.660873 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:32.661684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:32.662688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:32.663377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:32.663377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:32.663377 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:32.663710 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:32.664030 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:32.664232 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.664764 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 14:09:32.664764 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.664764 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:32.664764 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.664764 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.664764 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:32.664764 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.664764 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.665732 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.666102 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:32.666476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.666476 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:32.666793 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:32.667290 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:32.667808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:32.667808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:32.667808 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:37.387590 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:37.388284 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:37.388475 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:37.389688 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:37.427240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.427720 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.428047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:37.428394 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.428841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:37.580439 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:37.581435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:37.581435 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:37.582877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:37.582877 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:37.582877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.583456 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:37.583456 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.583456 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:37.583456 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.583456 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.583456 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:37.583456 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.583456 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.584872 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.585240 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:37.585600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.585600 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.586435 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.586881 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:37.587405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.587405 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:37.616115 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:37.616571 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:37.616676 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:37.616837 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:09:37.617375 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:37.617668 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.618363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:37.618363 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.618363 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:37.618363 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.618363 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.618363 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:37.618363 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.618363 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:37.619717 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.620811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:37.620811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.620811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:37.620811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.620811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.620811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:37.620811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.620811 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.621506 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:37.622204 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.622950 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:37.624111 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.625241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:37.625241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:37.625241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.625241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:37.625241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.625241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:37.625241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.625241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.625241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:37.625241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.625241 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:37.653451 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:37.654247 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:37.654575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:37.654888 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:37.655595 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:37.656054 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.656841 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:09:37.656841 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.656841 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:37.656841 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.656841 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.656841 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:37.656841 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.656841 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:37.657885 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:37.658514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.658514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:37.658514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.658514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:37.658514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.658514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.658514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:37.658514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.658514 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:37.659141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.659141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:37.659141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:37.659141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:37.659141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.659141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:37.659141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:37.659141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:37.659141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:37.659141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.659141 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:37.659685 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.660265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.660265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:37.660265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.660265 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.660724 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:37.661256 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.661812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.661812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:37.661812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.661812 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:37.662741 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:37.663373 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:37.663570 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:37.663915 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:37.664452 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:37.664815 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:37.665041 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.665821 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 14:09:37.665821 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.665821 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:37.665821 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.665821 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.665821 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:37.665821 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.665821 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.667554 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.667975 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:37.668383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.668383 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:37.668865 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:37.669336 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:37.669856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:37.669856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:37.669856 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:42.388575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:42.389455 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:42.389697 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:42.391142 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:42.400078 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG dbcounter [-] [106267] Writing DB stats neutron:SELECT=241 {{(pid=106267) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 21 14:09:42.430884 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:42.430884 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.431231 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:09:42.431231 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.431231 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:42.431231 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.431231 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.431231 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:42.431231 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.431231 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.432425 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.433047 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:42.433722 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:42.434432 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.435571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:42.435571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.435571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.435571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:42.435571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.435571 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:42.583941 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:42.585291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:42.585291 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:42.585811 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-40826ab9-f311-4f0d-921e-dee16be9cb42, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:42.586069 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, routes=[], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:42.586312 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.587146 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:42.587146 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.587146 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:42.587146 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.587146 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.587146 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:42.587146 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.587146 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.588775 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.589182 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:42.589584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.589584 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.589993 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 936, in check_router_default_route_empty_dst_ip
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.590550 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:42.591066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.591066 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:42.616905 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:42.617349 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:42.617422 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:42.617544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Setting fip-distributed flag in NB_Global to False {{(pid=106267) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1160}}
Apr 21 14:09:42.617890 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:42.618131 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.618672 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 21 14:09:42.618672 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.618672 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:42.618672 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.618672 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.618672 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:42.618672 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.618672 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [-] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:42.619768 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.620492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:42.620492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.620492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:42.620492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.620492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.620492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:42.620492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.620492 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.620877 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:42.621530 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [-] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.622115 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1164, in set_fip_distributed_flag
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:42.622617 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.623121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:42.623121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:42.623121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.623121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:42.623121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.623121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:42.623121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.623121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.623121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:42.623121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.623121 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:42.654444 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:42.656004 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:42.656004 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:42.656004 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:42.656476 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:42.656809 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.658410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Traceback (most recent call last):
Apr 21 14:09:42.658410 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.658410 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:42.658410 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.658410 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.658410 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:42.658410 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.658410 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 21 14:09:42.659055 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 21 14:09:42.659850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.659850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 21 14:09:42.659850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.659850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 21 14:09:42.659850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.659850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.659850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 21 14:09:42.659850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.659850 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 21 14:09:42.660358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.660358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:42.660358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:42.660358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:42.660358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.660358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:42.660358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:42.660358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:42.660358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:42.660358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.660358 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:42.660901 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-1217f9b9-b18d-4c59-be48-d80dee46b8f7 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:42.661771 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1225, in update_ha_failover
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:42.662637 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:42.663675 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:42.664327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:42.664327 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:42.664454 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:42.664969 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:42.665511 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:42.665749 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.666643 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Traceback (most recent call last):
Apr 21 14:09:42.666643 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.666643 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:42.666643 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.666643 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.666643 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:42.666643 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.666643 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.668313 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.668598 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:42.668895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.668895 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-a67d1a8e-b28f-4c06-8279-201e8743787d None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:42.670018 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 852, in update_mac_aging_settings
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:42.670420 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:42.670849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:42.670849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:42.670849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics 
Apr 21 14:09:46.466430 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:46.467672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:46.467672 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:46.477544 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:439}}
Apr 21 14:09:46.477685 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106267) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}}
Apr 21 14:09:46.477799 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Fixing resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:09:46.509346 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, resource={'id': 'b57c058f-054e-4a2a-980f-6bfda2111e90', 'name': 'private', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['6a3993e6-b5ba-4812-baa3-b25db5880cd9', 'c97deb27-a1d6-4e03-b489-95abde8fd791'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:29:32Z', 'updated_at': '2026-04-21T13:29:35Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 24380}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.509619 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b57c058f-054e-4a2a-980f-6bfda2111e90, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.510134 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-3f6c78fa-b671-412d-bb34-5426a40d8bff, external_ids_update=None, columns={'tag': 24380}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.510363 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.511230 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Traceback (most recent call last):
Apr 21 14:09:46.511230 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.511230 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:46.511230 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.511230 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.511230 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:46.511230 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.511230 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:46.512795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Failed to fix resource b57c058f-054e-4a2a-980f-6bfda2111e90 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.512795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:46.512795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:09:46.512795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:09:46.512795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:09:46.512795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:09:46.512795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 14:09:46.512795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:46.512795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.512795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.512795 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.513684 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.514883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:46.514883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.514883 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:46.514883 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Fixing resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:09:46.542804 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d0607710-681d-444d-be26-dbeca5a07b46, resource={'id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'name': 'public', 'tenant_id': 'cc293cfead344f7b90878133553b656f', 'project_id': 'cc293cfead344f7b90878133553b656f', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-21T13:29:38Z', 'updated_at': '2026-04-21T13:29:48Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.542981 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d0607710-681d-444d-be26-dbeca5a07b46, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.543324 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-19f04b40-ed99-482a-8774-a76892a716d1, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.543512 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.544120 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Traceback (most recent call last):
Apr 21 14:09:46.544120 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.544120 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:46.544120 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.544120 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.544120 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:46.544120 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.544120 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:46.545326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Failed to fix resource d0607710-681d-444d-be26-dbeca5a07b46 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.545326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:46.545326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:09:46.545326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:09:46.545326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:09:46.545326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:09:46.545326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 14:09:46.545326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:46.545326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.545326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.545326 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.546360 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.547462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:46.547462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.547462 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:46.547462 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Fixing resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:09:46.568188 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, resource={'id': '345e31fe-ede8-4dd0-823e-e81dcbc5bef0', 'name': 'admin_net', 'tenant_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'project_id': 'b53a9a23b5c24fe5bc22a9e228a333ca', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['bb271c4b-4172-418f-995c-f72abd439dbe'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:34:46Z', 'updated_at': '2026-04-21T13:34:47Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 22486}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.568402 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-345e31fe-ede8-4dd0-823e-e81dcbc5bef0, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.568575 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-c1b4e94e-49ea-4c1f-8edf-2f1433fdff16, external_ids_update=None, columns={'tag': 22486}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.568660 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.569013 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Traceback (most recent call last):
Apr 21 14:09:46.569013 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.569013 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:46.569013 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.569013 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.569013 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:46.569013 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.569013 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:46.569814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Failed to fix resource 345e31fe-ede8-4dd0-823e-e81dcbc5bef0 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.569814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:46.569814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:09:46.569814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:09:46.569814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:09:46.569814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:09:46.569814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 14:09:46.569814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:46.569814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.569814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.569814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.570387 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.571040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:46.571040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.571040 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:46.571040 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Fixing resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:09:46.595298 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, resource={'id': 'b364ad09-f70d-45d8-b9dd-8420f5f1c377', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['620e9559-2c15-4370-904f-f4677bc69542'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-21T13:35:24Z', 'updated_at': '2026-04-21T13:35:25Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 61006}, resource_type=networks, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.595509 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b364ad09-f70d-45d8-b9dd-8420f5f1c377, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.595932 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a100d4e8-7f11-40c9-8e4a-346c2c764f69, external_ids_update=None, columns={'tag': 61006}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.596042 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.596597 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Traceback (most recent call last):
Apr 21 14:09:46.596597 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.596597 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:46.596597 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.596597 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.596597 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:46.596597 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.596597 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:46.597700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Failed to fix resource b364ad09-f70d-45d8-b9dd-8420f5f1c377 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.597700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:46.597700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:09:46.597700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:09:46.597700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:09:46.597700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:09:46.597700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 21 14:09:46.597700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:46.597700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.597700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.597700 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.598544 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.599193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:46.599193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.599193 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:46.599193 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Fixing resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:09:46.671680 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, resource={'id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'project_id': '6a2c743728bd4bec95a3cf109e20ac04', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'enable_snat': True}, 'gw_port_id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-61m1f', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'external_fixed_ips': [{'ip_address': '172.24.5.49', 'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3'}, {'ip_address': '2001:db8::162', 'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 3, 'tenant_id': '6a2c743728bd4bec95a3cf109e20ac04'}, resource_type=routers, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.671896 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-06899d4c-a436-45e4-bed5-95dabe888f51, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-61m1f', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.672083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.672234 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d0607710-681d-444d-be26-dbeca5a07b46, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "48c36a92-d414-4b8c-a7f1-028c0fd5ae6d"')], if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.672362 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.673160 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Traceback (most recent call last):
Apr 21 14:09:46.673160 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.673160 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:46.673160 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.673160 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.673160 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:46.673160 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.673160 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:46.673431 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Unable to update router 06899d4c-a436-45e4-bed5-95dabe888f51. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.674830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Failed to fix resource 06899d4c-a436-45e4-bed5-95dabe888f51 (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.674830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:46.674830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:09:46.674830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:09:46.674830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:09:46.674830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:09:46.674830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 21 14:09:46.674830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 21 14:09:46.674830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 21 14:09:46.674830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 21 14:09:46.674830 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:46.675410 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:46.675849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:46.675849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:46.675849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.675849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:46.675849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.675849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.675849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:46.675849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.675849 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:46.675849 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Fixing resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports) at create/update {{(pid=106267) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:448}}
Apr 21 14:09:46.784714 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, resource={'id': '48c36a92-d414-4b8c-a7f1-028c0fd5ae6d', 'name': '', 'network_id': 'd0607710-681d-444d-be26-dbeca5a07b46', 'project_id': '', 'mac_address': 'fa:16:3e:fb:71:35', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '06899d4c-a436-45e4-bed5-95dabe888f51', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3', 'ip_address': '172.24.5.49'}, {'subnet_id': 'ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'ip_address': '2001:db8::162'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000168688', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-21T13:35:26Z', 'updated_at': '2026-04-21T13:35:28Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.784964 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'eb461636-5fef-46ea-85bc-4a8bc9466fe3 ca2ae363-6d7d-4b39-b330-44fb15b7cd49', 'neutron:network_name': 'neutron-d0607710-681d-444d-be26-dbeca5a07b46', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-06899d4c-a436-45e4-bed5-95dabe888f51'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.49/24', '2001:db8::162/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.785311 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, lrouter_port=lrp-48c36a92-d414-4b8c-a7f1-028c0fd5ae6d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:46.785547 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.786289 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Traceback (most recent call last):
Apr 21 14:09:46.786289 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.786289 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:46.786289 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.786289 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.786289 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:46.786289 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.786289 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:46.787784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] Maintenance task: Failed to fix resource 48c36a92-d414-4b8c-a7f1-028c0fd5ae6d (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.787784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:46.787784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 461, in check_for_inconsistencies
Apr 21 14:09:46.787784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 21 14:09:46.787784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 369, in _fix_create_update
Apr 21 14:09:46.787784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 21 14:09:46.787784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 21 14:09:46.787784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:46.787784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.787784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.787784 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:46.788395 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:46.789029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:46.789029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:46.789029 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:46.789029 np0000168688 neutron-ovn-maintenance-worker[106267]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-f5137723-a011-45f4-8d89-595f3b2e56b7 None None] OVN maintenance task check_for_inconsistencies finished in 0.321 seconds
Apr 21 14:09:47.390038 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106267) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 21 14:09:47.390867 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 21 14:09:47.391034 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106267) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 21 14:09:47.392083 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106267) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 21 14:09:47.435204 np0000168688 neutron-ovn-maintenance-worker[106267]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106267) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 21 14:09:47.435836 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:47.437403 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Traceback (most recent call last):
Apr 21 14:09:47.437403 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:47.437403 np0000168688 neutron-ovn-maintenance-worker[106267]:     txn.results.put(txn.do_commit())
Apr 21 14:09:47.437403 np0000168688 neutron-ovn-maintenance-worker[106267]:                     ^^^^^^^^^^^^^^^
Apr 21 14:09:47.437403 np0000168688 neutron-ovn-maintenance-worker[106267]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:47.437403 np0000168688 neutron-ovn-maintenance-worker[106267]:     raise RuntimeError(msg)
Apr 21 14:09:47.437403 np0000168688 neutron-ovn-maintenance-worker[106267]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:47.437403 np0000168688 neutron-ovn-maintenance-worker[106267]: 
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:47.438392 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:47.439166 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics [None req-dfcc410a-4378-421e-841d-17d0c441bb57 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics Traceback (most recent call last):
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     work()
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 21 14:09:47.439814 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 836, in check_fdb_aging_settings
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     next(self.gen)
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     with self.create_transaction(
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     self.result = self.commit()
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise result.ex
Apr 21 14:09:47.440667 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 21 14:09:47.442168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 21 14:09:47.442168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 21 14:09:47.442168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 21 14:09:47.442168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 21 14:09:47.442168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 21 14:09:47.442168 np0000168688 neutron-ovn-maintenance-worker[106267]: ERROR futurist.periodics